2025-12-04T09:19:38.6457183Z Current runner version: '2.330.0' 2025-12-04T09:19:38.6464238Z Runner name: 'i-01ba3f41dbfc68ae5' 2025-12-04T09:19:38.6465137Z Runner group name: 'default' 2025-12-04T09:19:38.6466198Z Machine name: 'ip-10-0-76-7' 2025-12-04T09:19:38.6469685Z ##[group]GITHUB_TOKEN Permissions 2025-12-04T09:19:38.6472542Z Contents: read 2025-12-04T09:19:38.6473238Z Metadata: read 2025-12-04T09:19:38.6473820Z ##[endgroup] 2025-12-04T09:19:38.6481919Z Secret source: Actions 2025-12-04T09:19:38.6482859Z Prepare workflow directory 2025-12-04T09:19:38.7114057Z Prepare all required actions 2025-12-04T09:19:38.7156759Z Getting action download info 2025-12-04T09:19:39.1159835Z Download action repository 'pytorch/test-infra@main' (SHA:39aa74d619174326f4e2fb0e216151c2f29d9ffd) 2025-12-04T09:19:41.5352363Z Download action repository 'pytorch/pytorch@main' (SHA:7716da9fb23f27a65b41f9f016a2afadf281c18f) 2025-12-04T09:19:57.3237757Z Download action repository 'actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065' (SHA:a26af69be951a213d495a4c3e4e4022e16d87065) 2025-12-04T09:19:57.6983781Z Download action repository 'aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722' (SHA:ececac1a45f3b08a01d2dd070d28d111c5fe6722) 2025-12-04T09:19:57.9672975Z Download action repository 'aws-actions/amazon-ecr-login@062b18b96a7aff071d4dc91bc00c4c1a7945b076' (SHA:062b18b96a7aff071d4dc91bc00c4c1a7945b076) 2025-12-04T09:19:58.1601561Z Download action repository 'seemethere/download-artifact-s3@1da556a7aa0a088e3153970611f6c432d58e80e6' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2025-12-04T09:19:58.4355401Z Download action repository 'seemethere/upload-artifact-s3@baba72d0712b404f646cebe0730933554ebce96a' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2025-12-04T09:19:58.7381638Z Getting action download info 2025-12-04T09:19:58.8578817Z Download action repository 'actions/checkout@v4' (SHA:34e114876b0b11c390a56381ad16ebd13914f8d5) 2025-12-04T09:19:59.1643540Z Getting action download info 2025-12-04T09:19:59.2950136Z Download action repository 'nick-fields/retry@v3.0.0' (SHA:7152eba30c6575329ac0576536151aca5a72780e) 2025-12-04T09:19:59.5535036Z Getting action download info 2025-12-04T09:19:59.6786001Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2025-12-04T09:19:59.9263957Z Getting action download info 2025-12-04T09:20:00.1162347Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/heads/main (ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32) 2025-12-04T09:20:00.1166879Z ##[group] Inputs 2025-12-04T09:20:00.1167278Z build-environment: linux-jammy-py3.10-gcc11-build 2025-12-04T09:20:00.1180547Z test-matrix: {"include": [{"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_avx2_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}]} 2025-12-04T09:20:00.1195637Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:20:00.1196711Z sync-tag: 2025-12-04T09:20:00.1197655Z timeout-minutes: 240 2025-12-04T09:20:00.1197961Z use-gha: 2025-12-04T09:20:00.1198204Z dashboard-tag: 2025-12-04T09:20:00.1198483Z s3-bucket: gha-artifacts 2025-12-04T09:20:00.1198804Z aws-role-to-assume: 2025-12-04T09:20:00.1199471Z disable-monitor: false 2025-12-04T09:20:00.1199833Z monitor-log-interval: 5 2025-12-04T09:20:00.1200203Z monitor-data-collect-interval: 1 2025-12-04T09:20:00.1200577Z ##[endgroup] 2025-12-04T09:20:00.1201796Z Complete job name: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:20:00.1992114Z A job started hook has been configured by the self-hosted runner administrator 2025-12-04T09:20:00.2111950Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2025-12-04T09:20:00.2122141Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:20:00.2123019Z ##[endgroup] 2025-12-04T09:20:01.9482993Z Runner Type: linux.10xlarge.avx2 2025-12-04T09:20:01.9483583Z Instance Type: m4.10xlarge 2025-12-04T09:20:01.9483891Z AMI Name: unknown 2025-12-04T09:20:01.9518006Z AMI ID: ami-08982f1c5bf93d976 2025-12-04T09:20:08.2043593Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2025-12-04T09:20:08.2044105Z with: 2025-12-04T09:20:08.2044701Z github-secret: *** 2025-12-04T09:20:08.2045523Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2025-12-04T09:20:08.2046417Z activate-with-label: false 2025-12-04T09:20:08.2046735Z label: with-ssh 2025-12-04T09:20:08.2047002Z remove-existing-keys: true 2025-12-04T09:20:08.2047552Z fail-silently: true 2025-12-04T09:20:08.2047826Z env: 2025-12-04T09:20:08.2048054Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:20:08.2048362Z ##[endgroup] 2025-12-04T09:20:08.3729705Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2025-12-04T09:20:08.3731307Z Not on pull request and ciflow reference could not be extracted, skipping adding ssh keys 2025-12-04T09:20:08.4113606Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2025-12-04T09:20:08.4114210Z with: 2025-12-04T09:20:08.4114486Z no-sudo: true 2025-12-04T09:20:08.4114782Z submodules: recursive 2025-12-04T09:20:08.4115089Z fetch-depth: 0 2025-12-04T09:20:08.4115363Z env: 2025-12-04T09:20:08.4115621Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:20:08.4115931Z ##[endgroup] 2025-12-04T09:20:08.4206028Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:20:08.4207201Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:20:08.4217709Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:20:08.4218168Z env: 2025-12-04T09:20:08.4218450Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:20:08.4218800Z ##[endgroup] 2025-12-04T09:20:08.4317509Z ##[group]Run # Use all available CPUs for fetching 2025-12-04T09:20:08.4318033Z # Use all available CPUs for fetching 2025-12-04T09:20:08.4318734Z cd "${GITHUB_WORKSPACE}" 2025-12-04T09:20:08.4319122Z git config --global fetch.parallel 0 2025-12-04T09:20:08.4319587Z git config --global submodule.fetchJobs 0 2025-12-04T09:20:08.4320260Z  2025-12-04T09:20:08.4320672Z # Clean workspace. The default checkout action should also do this, but 2025-12-04T09:20:08.4321244Z # do it here as well just in case 2025-12-04T09:20:08.4321643Z if [[ -d .git ]]; then 2025-12-04T09:20:08.4321976Z  if [ -z "${NO_SUDO}" ]; then 2025-12-04T09:20:08.4322348Z  sudo git clean -ffdx 2025-12-04T09:20:08.4322683Z  else 2025-12-04T09:20:08.4322961Z  git clean -ffdx 2025-12-04T09:20:08.4323263Z  fi 2025-12-04T09:20:08.4323523Z fi 2025-12-04T09:20:08.4329726Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:20:08.4330165Z env: 2025-12-04T09:20:08.4330498Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:20:08.4330840Z NO_SUDO: true 2025-12-04T09:20:08.4331093Z ##[endgroup] 2025-12-04T09:20:08.4463102Z ##[group]Run actions/checkout@v4 2025-12-04T09:20:08.4463456Z with: 2025-12-04T09:20:08.4463737Z ref: ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:20:08.4464144Z fetch-depth: 0 2025-12-04T09:20:08.4464427Z submodules: recursive 2025-12-04T09:20:08.4464722Z show-progress: false 2025-12-04T09:20:08.4465043Z repository: pytorch/pytorch 2025-12-04T09:20:08.4465557Z token: *** 2025-12-04T09:20:08.4465823Z ssh-strict: true 2025-12-04T09:20:08.4466079Z ssh-user: git 2025-12-04T09:20:08.4466362Z persist-credentials: true 2025-12-04T09:20:08.4466812Z clean: true 2025-12-04T09:20:08.4467261Z sparse-checkout-cone-mode: true 2025-12-04T09:20:08.4467621Z fetch-tags: false 2025-12-04T09:20:08.4467899Z lfs: false 2025-12-04T09:20:08.4468159Z set-safe-directory: true 2025-12-04T09:20:08.4468485Z env: 2025-12-04T09:20:08.4468744Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:20:08.4469041Z ##[endgroup] 2025-12-04T09:20:08.5721367Z Syncing repository: pytorch/pytorch 2025-12-04T09:20:08.5722963Z ##[group]Getting Git version info 2025-12-04T09:20:08.5723533Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2025-12-04T09:20:08.5724307Z [command]/usr/bin/git version 2025-12-04T09:20:08.5901149Z git version 2.50.1 2025-12-04T09:20:08.5937150Z ##[endgroup] 2025-12-04T09:20:08.5949070Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/1b6dcd6a-4720-4430-9053-79288ec9bcfb/.gitconfig' 2025-12-04T09:20:08.5968989Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/1b6dcd6a-4720-4430-9053-79288ec9bcfb' before making global git config changes 2025-12-04T09:20:08.5973561Z Adding repository directory to the temporary git global config as a safe directory 2025-12-04T09:20:08.5974742Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-12-04T09:20:08.6021131Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2025-12-04T09:20:08.6024903Z ##[group]Initializing the repository 2025-12-04T09:20:08.6029697Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-12-04T09:20:08.6097643Z hint: Using 'master' as the name for the initial branch. This default branch name 2025-12-04T09:20:08.6098430Z hint: is subject to change. To configure the initial branch name to use in all 2025-12-04T09:20:08.6099114Z hint: of your new repositories, which will suppress this warning, call: 2025-12-04T09:20:08.6099610Z hint: 2025-12-04T09:20:08.6099959Z hint: git config --global init.defaultBranch 2025-12-04T09:20:08.6100374Z hint: 2025-12-04T09:20:08.6101393Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2025-12-04T09:20:08.6102134Z hint: 'development'. The just-created branch can be renamed via this command: 2025-12-04T09:20:08.6102676Z hint: 2025-12-04T09:20:08.6102933Z hint: git branch -m 2025-12-04T09:20:08.6103256Z hint: 2025-12-04T09:20:08.6103706Z hint: Disable this message with "git config set advice.defaultBranchName false" 2025-12-04T09:20:08.6106179Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2025-12-04T09:20:08.6122185Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2025-12-04T09:20:08.6161757Z ##[endgroup] 2025-12-04T09:20:08.6162287Z ##[group]Disabling automatic garbage collection 2025-12-04T09:20:08.6164137Z [command]/usr/bin/git config --local gc.auto 0 2025-12-04T09:20:08.6202404Z ##[endgroup] 2025-12-04T09:20:08.6202884Z ##[group]Setting up auth 2025-12-04T09:20:08.6207963Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2025-12-04T09:20:08.6235898Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2025-12-04T09:20:08.6594236Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2025-12-04T09:20:08.6625034Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2025-12-04T09:20:08.6971902Z [command]/usr/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:20:08.7002788Z [command]/usr/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url 2025-12-04T09:20:08.7338147Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2025-12-04T09:20:08.7389179Z ##[endgroup] 2025-12-04T09:20:08.7389702Z ##[group]Fetching the repository 2025-12-04T09:20:08.7398545Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2025-12-04T09:21:03.8599943Z From https://github.com/pytorch/pytorch 2025-12-04T09:21:03.8601557Z * [new branch] 2.6.0.dev20241004+ -> origin/2.6.0.dev20241004+ 2025-12-04T09:21:03.8602824Z * [new branch] 2.9.1 -> origin/2.9.1 2025-12-04T09:21:03.8604146Z * [new branch] AaronWang04_addmmfusion_perftest -> origin/AaronWang04_addmmfusion_perftest 2025-12-04T09:21:03.8605628Z * [new branch] Flamefire-patch-1 -> origin/Flamefire-patch-1 2025-12-04T09:21:03.8607438Z * [new branch] HDCharles-2.6.0-release-notes -> origin/HDCharles-2.6.0-release-notes 2025-12-04T09:21:03.8608730Z * [new branch] HOPrintFunc -> origin/HOPrintFunc 2025-12-04T09:21:03.8609937Z * [new branch] IvanKobzarev/stack/1 -> origin/IvanKobzarev/stack/1 2025-12-04T09:21:03.8612631Z * [new branch] NicoshevSVE128 -> origin/NicoshevSVE128 2025-12-04T09:21:03.8614573Z * [new branch] PR-AOTInductorNoneBug -> origin/PR-AOTInductorNoneBug 2025-12-04T09:21:03.8616810Z * [new branch] PR-AOTInductorNoneBugFix -> origin/PR-AOTInductorNoneBugFix 2025-12-04T09:21:03.8618524Z * [new branch] PR-FixConfigsIssue -> origin/PR-FixConfigsIssue 2025-12-04T09:21:03.8620344Z * [new branch] PR-NoneBugFix-viable -> origin/PR-NoneBugFix-viable 2025-12-04T09:21:03.8622268Z * [new branch] PR-ResetToZero -> origin/PR-ResetToZero 2025-12-04T09:21:03.8624356Z * [new branch] Update-Flash-Packaging -> origin/Update-Flash-Packaging 2025-12-04T09:21:03.8626055Z * [new branch] VLA_exp -> origin/VLA_exp 2025-12-04T09:21:03.8629024Z * [new branch] activation_bench -> origin/activation_bench 2025-12-04T09:21:03.8630572Z * [new branch] addmm-heuristic -> origin/addmm-heuristic 2025-12-04T09:21:03.8633486Z * [new branch] adi/onednn_aarch64 -> origin/adi/onednn_aarch64 2025-12-04T09:21:03.8635034Z * [new branch] adi/test -> origin/adi/test 2025-12-04T09:21:03.8637256Z * [new branch] adi/test_bgemm -> origin/adi/test_bgemm 2025-12-04T09:21:03.8639360Z * [new branch] adi/test_m8g -> origin/adi/test_m8g 2025-12-04T09:21:03.8641270Z * [new branch] adi/test_onednn -> origin/adi/test_onednn 2025-12-04T09:21:03.8643132Z * [new branch] adi/test_onednn_v3.9 -> origin/adi/test_onednn_v3.9 2025-12-04T09:21:03.8645058Z * [new branch] adi/test_presve_change -> origin/adi/test_presve_change 2025-12-04T09:21:03.8646772Z * [new branch] adi/test_timm -> origin/adi/test_timm 2025-12-04T09:21:03.8649388Z * [new branch] adi/testpresve_change -> origin/adi/testpresve_change 2025-12-04T09:21:03.8658010Z * [new branch] aditew01/test/vec_bf16 -> origin/aditew01/test/vec_bf16 2025-12-04T09:21:03.8659511Z * [new branch] ah-globalfeedback-hook -> origin/ah-globalfeedback-hook 2025-12-04T09:21:03.8661746Z * [new branch] albanD-patch-1 -> origin/albanD-patch-1 2025-12-04T09:21:03.8663846Z * [new branch] also-surround-shimh -> origin/also-surround-shimh 2025-12-04T09:21:03.8666652Z * [new branch] angelayi/aot_compile -> origin/angelayi/aot_compile 2025-12-04T09:21:03.8668794Z * [new branch] angelayi/aoti_additional_files -> origin/angelayi/aoti_additional_files 2025-12-04T09:21:03.8670554Z * [new branch] angelayi/benchmark -> origin/angelayi/benchmark 2025-12-04T09:21:03.8672728Z * [new branch] angelayi/change_pytree_serialization -> origin/angelayi/change_pytree_serialization 2025-12-04T09:21:03.8674467Z * [new branch] angelayi/cpp_loader -> origin/angelayi/cpp_loader 2025-12-04T09:21:03.8676149Z * [new branch] angelayi/inductor_const -> origin/angelayi/inductor_const 2025-12-04T09:21:03.8677656Z * [new branch] angelayi/lstm -> origin/angelayi/lstm 2025-12-04T09:21:03.8680230Z * [new branch] angelayi/no_so_weight -> origin/angelayi/no_so_weight 2025-12-04T09:21:03.8682250Z * [new branch] angelayi/scan_layers -> origin/angelayi/scan_layers 2025-12-04T09:21:03.8683983Z * [new branch] angelayi/side_eff -> origin/angelayi/side_eff 2025-12-04T09:21:03.8685658Z * [new branch] angelayi/state_dict -> origin/angelayi/state_dict 2025-12-04T09:21:03.8687318Z * [new branch] angelayi/symint_input -> origin/angelayi/symint_input 2025-12-04T09:21:03.8689382Z * [new branch] angelayi/symm_mem -> origin/angelayi/symm_mem 2025-12-04T09:21:03.8690728Z * [new branch] angelayi/test_cpp -> origin/angelayi/test_cpp 2025-12-04T09:21:03.8692297Z * [new branch] angelayi/torch_size -> origin/angelayi/torch_size 2025-12-04T09:21:03.8693901Z * [new branch] annotate_assert -> origin/annotate_assert 2025-12-04T09:21:03.8695549Z * [new branch] annotate_fallback_kernel -> origin/annotate_fallback_kernel 2025-12-04T09:21:03.8697273Z * [new branch] annotation_deepcopy -> origin/annotation_deepcopy 2025-12-04T09:21:03.8698875Z * [new branch] annotation_dynamo -> origin/annotation_dynamo 2025-12-04T09:21:03.8700419Z * [new branch] aot_eager_stack_trace -> origin/aot_eager_stack_trace 2025-12-04T09:21:03.8702953Z * [new branch] aoti-cuda-alloc -> origin/aoti-cuda-alloc 2025-12-04T09:21:03.8704515Z * [new branch] aoti_const_device -> origin/aoti_const_device 2025-12-04T09:21:03.8706237Z * [new branch] aoti_fqn_name_interface -> origin/aoti_fqn_name_interface 2025-12-04T09:21:03.8708089Z * [new branch] aoti_package_weights_binary -> origin/aoti_package_weights_binary 2025-12-04T09:21:03.8709584Z * [new branch] aoti_target_windows -> origin/aoti_target_windows 2025-12-04T09:21:03.8712352Z * [new branch] arsh/feat/inductor_check_profiling -> origin/arsh/feat/inductor_check_profiling 2025-12-04T09:21:03.8714222Z * [new branch] async_tp -> origin/async_tp 2025-12-04T09:21:03.8715726Z * [new branch] atalman-inductor-perf-cu124 -> origin/atalman-inductor-perf-cu124 2025-12-04T09:21:03.8717017Z * [new branch] atalman-inductor-perf-cu124.1 -> origin/atalman-inductor-perf-cu124.1 2025-12-04T09:21:03.8718388Z * [new branch] atalman-patch-2 -> origin/atalman-patch-2 2025-12-04T09:21:03.8719784Z * [new branch] atalman-patch-3 -> origin/atalman-patch-3 2025-12-04T09:21:03.8721137Z * [new branch] atalman-patch-4 -> origin/atalman-patch-4 2025-12-04T09:21:03.8722632Z * [new branch] atalman-patch-5 -> origin/atalman-patch-5 2025-12-04T09:21:03.8724400Z * [new branch] atalman-patch-6 -> origin/atalman-patch-6 2025-12-04T09:21:03.8726267Z * [new branch] atalman-patch-7 -> origin/atalman-patch-7 2025-12-04T09:21:03.8727936Z * [new branch] atalman-patch-8 -> origin/atalman-patch-8 2025-12-04T09:21:03.8729599Z * [new branch] atalman_inductor_2.3.1 -> origin/atalman_inductor_2.3.1 2025-12-04T09:21:03.8731215Z * [new branch] atalman_inductor_2.4.0 -> origin/atalman_inductor_2.4.0 2025-12-04T09:21:03.8732900Z * [new branch] atalman_inductor_2.4.x -> origin/atalman_inductor_2.4.x 2025-12-04T09:21:03.8734621Z * [new branch] attention_benchmarking_clean -> origin/attention_benchmarking_clean 2025-12-04T09:21:03.8736581Z * [new branch] bahuang/dt_fix_scalar_add -> origin/bahuang/dt_fix_scalar_add 2025-12-04T09:21:03.8738216Z * [new branch] bahuang/fix_debug_mode -> origin/bahuang/fix_debug_mode 2025-12-04T09:21:03.8739788Z * [new branch] bahuang/fix_expand -> origin/bahuang/fix_expand 2025-12-04T09:21:03.8741330Z * [new branch] bahuang/test -> origin/bahuang/test 2025-12-04T09:21:03.8743579Z * [new branch] base/1.5 -> origin/base/1.5 2025-12-04T09:21:03.8745405Z * [new branch] batching_sdpa_efficient_attention -> origin/batching_sdpa_efficient_attention 2025-12-04T09:21:03.8747468Z * [new branch] bench_scaled_mm_ops -> origin/bench_scaled_mm_ops 2025-12-04T09:21:03.8749132Z * [new branch] benchmark-updates -> origin/benchmark-updates 2025-12-04T09:21:03.8750745Z * [new branch] benchmarking-script -> origin/benchmarking-script 2025-12-04T09:21:03.8752800Z * [new branch] bertmaher/pinbump26 -> origin/bertmaher/pinbump26 2025-12-04T09:21:03.8754889Z * [new branch] bertrand/cutlass -> origin/bertrand/cutlass 2025-12-04T09:21:03.8757001Z * [new branch] bf/bug-static-input -> origin/bf/bug-static-input 2025-12-04T09:21:03.8758619Z * [new branch] bf/cg-backend -> origin/bf/cg-backend 2025-12-04T09:21:03.8760342Z * [new branch] bf/cg-nccl-test -> origin/bf/cg-nccl-test 2025-12-04T09:21:03.8761795Z * [new branch] bf/cg-remove-check -> origin/bf/cg-remove-check 2025-12-04T09:21:03.8763496Z * [new branch] bf/clean-torchbench-hf -> origin/bf/clean-torchbench-hf 2025-12-04T09:21:03.8764992Z * [new branch] bf/combo-debug-log -> origin/bf/combo-debug-log 2025-12-04T09:21:03.8766552Z * [new branch] bf/cudagraph -> origin/bf/cudagraph 2025-12-04T09:21:03.8768626Z * [new branch] bf/cudagraph-disable-input-mutation -> origin/bf/cudagraph-disable-input-mutation 2025-12-04T09:21:03.8770506Z * [new branch] bf/cudagraph-enable-input-mutation-support-benchmark -> origin/bf/cudagraph-enable-input-mutation-support-benchmark 2025-12-04T09:21:03.8771773Z * [new branch] bf/cudagraph-partition -> origin/bf/cudagraph-partition 2025-12-04T09:21:03.8773441Z * [new branch] bf/donated-buffer-bench -> origin/bf/donated-buffer-bench 2025-12-04T09:21:03.8775101Z * [new branch] bf/dynamo-partition -> origin/bf/dynamo-partition 2025-12-04T09:21:03.8785929Z * [new branch] bf/lite -> origin/bf/lite 2025-12-04T09:21:03.8787289Z * [new branch] bf/pa-non-divisible -> origin/bf/pa-non-divisible 2025-12-04T09:21:03.8788733Z * [new branch] bf/partition-cache-free-symbols -> origin/bf/partition-cache-free-symbols 2025-12-04T09:21:03.8790254Z * [new branch] bf/partition-memory-plan -> origin/bf/partition-memory-plan 2025-12-04T09:21:03.8791621Z * [new branch] bf/partition-move-cpu -> origin/bf/partition-move-cpu 2025-12-04T09:21:03.8793003Z * [new branch] bf/partition-view-fallback -> origin/bf/partition-view-fallback 2025-12-04T09:21:03.8794453Z * [new branch] bf/remove-check-55b0c39d -> origin/bf/remove-check-55b0c39d 2025-12-04T09:21:03.8795727Z * [new branch] bf/timm-nov-26-2025 -> origin/bf/timm-nov-26-2025 2025-12-04T09:21:03.8797028Z * [new branch] bf/transformer-pin-4-57-3 -> origin/bf/transformer-pin-4-57-3 2025-12-04T09:21:03.8798570Z * [new branch] bisect_perf_hf_T5_3acc6eac492 -> origin/bisect_perf_hf_T5_3acc6eac492 2025-12-04T09:21:03.8799933Z * [new branch] bisect_perf_hf_T5_3fcf66f61fb -> origin/bisect_perf_hf_T5_3fcf66f61fb 2025-12-04T09:21:03.8801985Z * [new branch] bisect_perf_hf_T5_4009d154129 -> origin/bisect_perf_hf_T5_4009d154129 2025-12-04T09:21:03.8803354Z * [new branch] bisect_perf_hf_T5_40d0740e73d -> origin/bisect_perf_hf_T5_40d0740e73d 2025-12-04T09:21:03.8804714Z * [new branch] bisect_perf_hf_T5_5268754e -> origin/bisect_perf_hf_T5_5268754e 2025-12-04T09:21:03.8806044Z * [new branch] bisect_perf_hf_T5_7d89a8d385c -> origin/bisect_perf_hf_T5_7d89a8d385c 2025-12-04T09:21:03.8807566Z * [new branch] bisect_perf_hf_T5_b7a25c1ee7c -> origin/bisect_perf_hf_T5_b7a25c1ee7c 2025-12-04T09:21:03.8808968Z * [new branch] bisect_perf_hf_T5_c25b201583f -> origin/bisect_perf_hf_T5_c25b201583f 2025-12-04T09:21:03.8810360Z * [new branch] bisect_perf_hf_T5_c93e57efac0 -> origin/bisect_perf_hf_T5_c93e57efac0 2025-12-04T09:21:03.8811920Z * [new branch] bisect_perf_hf_T5_ca9813ea149 -> origin/bisect_perf_hf_T5_ca9813ea149 2025-12-04T09:21:03.8813404Z * [new branch] bisect_perf_hf_T5_d65f194a -> origin/bisect_perf_hf_T5_d65f194a 2025-12-04T09:21:03.8814693Z * [new branch] bisect_perf_hf_T5_da94ab0b -> origin/bisect_perf_hf_T5_da94ab0b 2025-12-04T09:21:03.8816087Z * [new branch] bisect_perf_hf_T5_da94ab0b_new -> origin/bisect_perf_hf_T5_da94ab0b_new 2025-12-04T09:21:03.8817447Z * [new branch] bisect_perf_hf_T5_db4e8a1d8a8 -> origin/bisect_perf_hf_T5_db4e8a1d8a8 2025-12-04T09:21:03.8818836Z * [new branch] bisect_perf_hf_T5_e0d97e936a2 -> origin/bisect_perf_hf_T5_e0d97e936a2 2025-12-04T09:21:03.8820195Z * [new branch] bisect_perf_hf_T5_f23621ec563 -> origin/bisect_perf_hf_T5_f23621ec563 2025-12-04T09:21:03.8821842Z * [new branch] brister/fx_device_type -> origin/brister/fx_device_type 2025-12-04T09:21:03.8823769Z * [new branch] brister/test_inductor_all_fx -> origin/brister/test_inductor_all_fx 2025-12-04T09:21:03.8825671Z * [new branch] brister/tiled_reduction_no_numel_check -> origin/brister/tiled_reduction_no_numel_check 2025-12-04T09:21:03.8827518Z * [new branch] bwd-backup -> origin/bwd-backup 2025-12-04T09:21:03.8829860Z * [new branch] c57382a49 -> origin/c57382a49 2025-12-04T09:21:03.8831572Z * [new branch] ca_0431d47eaa -> origin/ca_0431d47eaa 2025-12-04T09:21:03.8833453Z * [new branch] ca_fix_0431d47eaa -> origin/ca_fix_0431d47eaa 2025-12-04T09:21:03.8836440Z * [new branch] camyllh/test_setup_hooks_push -> origin/camyllh/test_setup_hooks_push 2025-12-04T09:21:03.8843487Z * [new branch] cccclai-patch-1 -> origin/cccclai-patch-1 2025-12-04T09:21:03.8845780Z * [new branch] cherry-pick-159969-by-pytorch_bot_bot_ -> origin/cherry-pick-159969-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8847655Z * [new branch] cherry-pick-160586-by-pytorch_bot_bot_ -> origin/cherry-pick-160586-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8849906Z * [new branch] cherry-pick-162208-by-pytorch_bot_bot_ -> origin/cherry-pick-162208-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8851721Z * [new branch] cherry-pick-163169-by-pytorch_bot_bot_ -> origin/cherry-pick-163169-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8853616Z * [new branch] cherry-pick-165086-by-pytorch_bot_bot_ -> origin/cherry-pick-165086-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8855778Z * [new branch] cherry-pick-165514-by-pytorch_bot_bot_ -> origin/cherry-pick-165514-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8857579Z * [new branch] cherry-pick-165601-by-pytorch_bot_bot_ -> origin/cherry-pick-165601-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8859498Z * [new branch] cherry-pick-165667-by-pytorch_bot_bot_ -> origin/cherry-pick-165667-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8861519Z * [new branch] cherry-pick-165815-by-pytorch_bot_bot_ -> origin/cherry-pick-165815-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8863506Z * [new branch] cherry-pick-165922-by-pytorch_bot_bot_ -> origin/cherry-pick-165922-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8865327Z * [new branch] cherry-pick-166148-by-pytorch_bot_bot_ -> origin/cherry-pick-166148-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8867452Z * [new branch] cherry-pick-166181-by-pytorch_bot_bot_ -> origin/cherry-pick-166181-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8869345Z * [new branch] cherry-pick-166404-by-pytorch_bot_bot_ -> origin/cherry-pick-166404-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8870918Z * [new branch] cherry-pick-166427-by-pytorch_bot_bot_ -> origin/cherry-pick-166427-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8872810Z * [new branch] cherry-pick-166480-by-pytorch_bot_bot_ -> origin/cherry-pick-166480-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8874324Z * [new branch] cherry-pick-166570-by-pytorch_bot_bot_ -> origin/cherry-pick-166570-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8876072Z * [new branch] cherry-pick-166993-by-pytorch_bot_bot_ -> origin/cherry-pick-166993-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8877782Z * [new branch] cherry-pick-167111-by-pytorch_bot_bot_ -> origin/cherry-pick-167111-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8879807Z * [new branch] cherry-pick-167478-by-pytorch_bot_bot_ -> origin/cherry-pick-167478-by-pytorch_bot_bot_ 2025-12-04T09:21:03.8881050Z * [new branch] cherry_pick_166036_166040 -> origin/cherry_pick_166036_166040 2025-12-04T09:21:03.8882722Z * [new branch] cherry_pick_166457 -> origin/cherry_pick_166457 2025-12-04T09:21:03.8884437Z * [new branch] cherrypick_166338 -> origin/cherrypick_166338 2025-12-04T09:21:03.8886053Z * [new branch] cherrypick_166458 -> origin/cherrypick_166458 2025-12-04T09:21:03.8887642Z * [new branch] cherrypick_166586 -> origin/cherrypick_166586 2025-12-04T09:21:03.8889192Z * [new branch] cherrypick_166956 -> origin/cherrypick_166956 2025-12-04T09:21:03.8890795Z * [new branch] ci_attn -> origin/ci_attn 2025-12-04T09:21:03.8892408Z * [new branch] codex-testing -> origin/codex-testing 2025-12-04T09:21:03.8894875Z * [new branch] codex/add-check_memory_overlap-helper-functions -> origin/codex/add-check_memory_overlap-helper-functions 2025-12-04T09:21:03.8896229Z * [new branch] codex/fix-issue-121219-in-pytorch -> origin/codex/fix-issue-121219-in-pytorch 2025-12-04T09:21:03.8898362Z * [new branch] codex/investigate-segfaults-in-get_tensor_storage_id -> origin/codex/investigate-segfaults-in-get_tensor_storage_id 2025-12-04T09:21:03.8903317Z * [new branch] codex/refactor-lintrunner-config-to-use-uv-run -> origin/codex/refactor-lintrunner-config-to-use-uv-run 2025-12-04T09:21:03.8907112Z * [new branch] compatiblpy39util -> origin/compatiblpy39util 2025-12-04T09:21:03.8908819Z * [new branch] cond_hop_device -> origin/cond_hop_device 2025-12-04T09:21:03.8910407Z * [new branch] context_test -> origin/context_test 2025-12-04T09:21:03.8912818Z * [new branch] copilot/code-style-cleanup-python-pip -> origin/copilot/code-style-cleanup-python-pip 2025-12-04T09:21:03.8914754Z * [new branch] cpio/fix_new_ami_tests -> origin/cpio/fix_new_ami_tests 2025-12-04T09:21:03.8916520Z * [new branch] cpp-docs-dependency-upgrade -> origin/cpp-docs-dependency-upgrade 2025-12-04T09:21:03.8918977Z * [new branch] crpa/typo-in-inductor_comm_lowering -> origin/crpa/typo-in-inductor_comm_lowering 2025-12-04T09:21:03.8920875Z * [new branch] csl/always_produce_xml -> origin/csl/always_produce_xml 2025-12-04T09:21:03.8922403Z * [new branch] csl/build_test_more_procs -> origin/csl/build_test_more_procs 2025-12-04T09:21:03.8923989Z * [new branch] csl/build_test_more_procs2 -> origin/csl/build_test_more_procs2 2025-12-04T09:21:03.8925542Z * [new branch] csl/clean_up -> origin/csl/clean_up 2025-12-04T09:21:03.8927171Z * [new branch] csl/fix_retry_segfault_exit -> origin/csl/fix_retry_segfault_exit 2025-12-04T09:21:03.8928609Z * [new branch] csl/katex -> origin/csl/katex 2025-12-04T09:21:03.8930460Z * [new branch] csl/larger_runner -> origin/csl/larger_runner 2025-12-04T09:21:03.8932300Z * [new branch] csl/lint_testing -> origin/csl/lint_testing 2025-12-04T09:21:03.8934190Z * [new branch] csl/lint_thing -> origin/csl/lint_thing 2025-12-04T09:21:03.8935914Z * [new branch] csl/lintrunner_stuff -> origin/csl/lintrunner_stuff 2025-12-04T09:21:03.8937522Z * [new branch] csl/manually_gen_json -> origin/csl/manually_gen_json 2025-12-04T09:21:03.8939096Z * [new branch] csl/mps_sharding -> origin/csl/mps_sharding 2025-12-04T09:21:03.8940773Z * [new branch] csl/multistage_docker -> origin/csl/multistage_docker 2025-12-04T09:21:03.8942367Z * [new branch] csl/print_timing -> origin/csl/print_timing 2025-12-04T09:21:03.8943910Z * [new branch] csl/remove_experiment -> origin/csl/remove_experiment 2025-12-04T09:21:03.8945532Z * [new branch] csl/remove_maybe_unused_var -> origin/csl/remove_maybe_unused_var 2025-12-04T09:21:03.8947653Z * [new branch] csl/remove_repo_specific_autolabel -> origin/csl/remove_repo_specific_autolabel 2025-12-04T09:21:03.8949250Z * [new branch] csl/remove_run_parallel -> origin/csl/remove_run_parallel 2025-12-04T09:21:03.8950761Z * [new branch] csl/remove_unused_vars -> origin/csl/remove_unused_vars 2025-12-04T09:21:03.8952712Z * [new branch] csl/revert_open -> origin/csl/revert_open 2025-12-04T09:21:03.8954436Z * [new branch] csl/skip_build -> origin/csl/skip_build 2025-12-04T09:21:03.8956098Z * [new branch] csl/smaller_avx_amx_runenrs -> origin/csl/smaller_avx_amx_runenrs 2025-12-04T09:21:03.8957602Z * [new branch] csl/td_job_level -> origin/csl/td_job_level 2025-12-04T09:21:03.8959418Z * [new branch] csl/test_cuda_build_large_runner -> origin/csl/test_cuda_build_large_runner 2025-12-04T09:21:03.8961119Z * [new branch] csl/test_owners_autograd_dispatch_nn -> origin/csl/test_owners_autograd_dispatch_nn 2025-12-04T09:21:03.8962787Z * [new branch] csl/test_owners_higher_confidence -> origin/csl/test_owners_higher_confidence 2025-12-04T09:21:03.8971352Z * [new branch] csl/upload_json_running -> origin/csl/upload_json_running 2025-12-04T09:21:03.8972688Z * [new branch] csl/win_sccache -> origin/csl/win_sccache 2025-12-04T09:21:03.8973792Z * [new branch] csl/xml_stuff -> origin/csl/xml_stuff 2025-12-04T09:21:03.8974870Z * [new branch] cublasrelax2 -> origin/cublasrelax2 2025-12-04T09:21:03.8976047Z * [new branch] cuda_mempool -> origin/cuda_mempool 2025-12-04T09:21:03.8977161Z * [new branch] custom_lowering_dict -> origin/custom_lowering_dict 2025-12-04T09:21:03.8978452Z * [new branch] d4l3k/debug_plane_frtrace -> origin/d4l3k/debug_plane_frtrace 2025-12-04T09:21:03.8979625Z * [new branch] daxia6/2.8o3 -> origin/daxia6/2.8o3 2025-12-04T09:21:03.8980672Z * [new branch] debug-guard -> origin/debug-guard 2025-12-04T09:21:03.8981732Z * [new branch] delete-quant-docs -> origin/delete-quant-docs 2025-12-04T09:21:03.8986646Z * [new branch] dependabot/pip/dot-ci/docker/ci_commit_pins/main/transformers-4.57.0 -> origin/dependabot/pip/dot-ci/docker/ci_commit_pins/main/transformers-4.57.0 2025-12-04T09:21:03.8989917Z * [new branch] dependabot/pip/dot-ci/docker/ci_commit_pins/main/transformers-4.57.1 -> origin/dependabot/pip/dot-ci/docker/ci_commit_pins/main/transformers-4.57.1 2025-12-04T09:21:03.8992221Z * [new branch] desertfire/test_cpp_wrapper -> origin/desertfire/test_cpp_wrapper 2025-12-04T09:21:03.8993949Z * [new branch] desertfire/triton-cpu-for-aarch64 -> origin/desertfire/triton-cpu-for-aarch64 2025-12-04T09:21:03.8995449Z * [new branch] dev/dhruva/flex_attn_opt -> origin/dev/dhruva/flex_attn_opt 2025-12-04T09:21:03.8998534Z * [new branch] dev/joona/MPSNDArrayAdd -> origin/dev/joona/MPSNDArrayAdd 2025-12-04T09:21:03.9000703Z * [new branch] dev/joona/Unranked -> origin/dev/joona/Unranked 2025-12-04T09:21:03.9004055Z * [new branch] dev/joona/cat -> origin/dev/joona/cat 2025-12-04T09:21:03.9005847Z * [new branch] dev/joona/embeddingbag -> origin/dev/joona/embeddingbag 2025-12-04T09:21:03.9007783Z * [new branch] dev/joona/fix_sdpa_memtest -> origin/dev/joona/fix_sdpa_memtest 2025-12-04T09:21:03.9010425Z * [new branch] dev/joona/getTensorsString -> origin/dev/joona/getTensorsString 2025-12-04T09:21:03.9012729Z * [new branch] dev/joona/mps_linear_macos14 -> origin/dev/joona/mps_linear_macos14 2025-12-04T09:21:03.9015563Z * [new branch] dev/joona/scalar_clamp -> origin/dev/joona/scalar_clamp 2025-12-04T09:21:03.9018127Z * [new branch] dev/joona/sdpa -> origin/dev/joona/sdpa 2025-12-04T09:21:03.9020901Z * [new branch] dev/joona/sdpa_api -> origin/dev/joona/sdpa_api 2025-12-04T09:21:03.9023128Z * [new branch] dev/joona/type_inf -> origin/dev/joona/type_inf 2025-12-04T09:21:03.9028007Z * [new branch] dev/joona/ulpAssertClose -> origin/dev/joona/ulpAssertClose 2025-12-04T09:21:03.9031045Z * [new branch] dev/joona/upsize3d -> origin/dev/joona/upsize3d 2025-12-04T09:21:03.9032225Z * [new branch] disp_counter -> origin/disp_counter 2025-12-04T09:21:03.9033394Z * [new branch] divyanshk-patch-1 -> origin/divyanshk-patch-1 2025-12-04T09:21:03.9034497Z * [new branch] docs -> origin/docs 2025-12-04T09:21:03.9035540Z * [new branch] documentation -> origin/documentation 2025-12-04T09:21:03.9037009Z * [new branch] eager_model_benchmarks -> origin/eager_model_benchmarks 2025-12-04T09:21:03.9040086Z * [new branch] embg/test_inductor_ci_control -> origin/embg/test_inductor_ci_control 2025-12-04T09:21:03.9041604Z * [new branch] embg/triton_l2_prefetch_128B -> origin/embg/triton_l2_prefetch_128B 2025-12-04T09:21:03.9043274Z * [new branch] embg/triton_l2_prefetch_256B -> origin/embg/triton_l2_prefetch_256B 2025-12-04T09:21:03.9045090Z * [new branch] eqy-patch-1 -> origin/eqy-patch-1 2025-12-04T09:21:03.9047216Z * [new branch] eqy-patch-2 -> origin/eqy-patch-2 2025-12-04T09:21:03.9049267Z * [new branch] eqy-patch-3 -> origin/eqy-patch-3 2025-12-04T09:21:03.9051138Z * [new branch] eqy-patch-4 -> origin/eqy-patch-4 2025-12-04T09:21:03.9053204Z * [new branch] eqy-patch-5 -> origin/eqy-patch-5 2025-12-04T09:21:03.9054954Z * [new branch] eqy-patch-6 -> origin/eqy-patch-6 2025-12-04T09:21:03.9057776Z * [new branch] exclamaforte/amd-ma -> origin/exclamaforte/amd-ma 2025-12-04T09:21:03.9059748Z * [new branch] exclamaforte/combo-kernels-perf-run -> origin/exclamaforte/combo-kernels-perf-run 2025-12-04T09:21:03.9061217Z * [new branch] exclamaforte/do_bench_refactor -> origin/exclamaforte/do_bench_refactor 2025-12-04T09:21:03.9062856Z * [new branch] exclamaforte/enable-mem-dep-fusion -> origin/exclamaforte/enable-mem-dep-fusion 2025-12-04T09:21:03.9064657Z * [new branch] exclamaforte/fix-exhaustive-autotuning -> origin/exclamaforte/fix-exhaustive-autotuning 2025-12-04T09:21:03.9066615Z * [new branch] exclamaforte/fix-trace-parsing-fx-svg -> origin/exclamaforte/fix-trace-parsing-fx-svg 2025-12-04T09:21:03.9068822Z * [new branch] exclamaforte/force-pointwise-cat-perf-run -> origin/exclamaforte/force-pointwise-cat-perf-run 2025-12-04T09:21:03.9070239Z * [new branch] exclamaforte/fusion-data -> origin/exclamaforte/fusion-data 2025-12-04T09:21:03.9072209Z * [new branch] exclamaforte/gemm-benchmark-run -> origin/exclamaforte/gemm-benchmark-run 2025-12-04T09:21:03.9073680Z * [new branch] exclamaforte/gemm-export-model -> origin/exclamaforte/gemm-export-model 2025-12-04T09:21:03.9075362Z * [new branch] exclamaforte/gemm-model -> origin/exclamaforte/gemm-model 2025-12-04T09:21:03.9077252Z * [new branch] exclamaforte/gemm-model-all-data-collection -> origin/exclamaforte/gemm-model-all-data-collection 2025-12-04T09:21:03.9078637Z * [new branch] exclamaforte/gemm-to-amd -> origin/exclamaforte/gemm-to-amd 2025-12-04T09:21:03.9080365Z * [new branch] exclamaforte/just-gemm-model -> origin/exclamaforte/just-gemm-model 2025-12-04T09:21:03.9082153Z * [new branch] exclamaforte/just-gemm-model-no-refactor -> origin/exclamaforte/just-gemm-model-no-refactor 2025-12-04T09:21:03.9083627Z * [new branch] exclamaforte/profile-diff-algo -> origin/exclamaforte/profile-diff-algo 2025-12-04T09:21:03.9085333Z * [new branch] exclamaforte/profiler-visualization -> origin/exclamaforte/profiler-visualization 2025-12-04T09:21:03.9086932Z * [new branch] exclamaforte/test_cpp_wrapper_mode -> origin/exclamaforte/test_cpp_wrapper_mode 2025-12-04T09:21:03.9095184Z * [new branch] exclamaforte/update-autotune-configs -> origin/exclamaforte/update-autotune-configs 2025-12-04T09:21:03.9096189Z * [new branch] exclamaforte/update-autotune-configs-2 -> origin/exclamaforte/update-autotune-configs-2 2025-12-04T09:21:03.9096976Z * [new branch] exec -> origin/exec 2025-12-04T09:21:03.9098630Z * [new branch] experimental-mosaic -> origin/experimental-mosaic 2025-12-04T09:21:03.9100338Z * [new branch] export-D61047529 -> origin/export-D61047529 2025-12-04T09:21:03.9102809Z * [new branch] export-D71412006 -> origin/export-D71412006 2025-12-04T09:21:03.9105107Z * [new branch] export-D73042989 -> origin/export-D73042989 2025-12-04T09:21:03.9107353Z * [new branch] export-D78957093 -> origin/export-D78957093 2025-12-04T09:21:03.9109470Z * [new branch] export-D78996107 -> origin/export-D78996107 2025-12-04T09:21:03.9111106Z * [new branch] export-D80823877 -> origin/export-D80823877 2025-12-04T09:21:03.9112886Z * [new branch] export-D80958642 -> origin/export-D80958642 2025-12-04T09:21:03.9114514Z * [new branch] export-D81054193 -> origin/export-D81054193 2025-12-04T09:21:03.9116110Z * [new branch] export-D81204584 -> origin/export-D81204584 2025-12-04T09:21:03.9117716Z * [new branch] export-D81429090 -> origin/export-D81429090 2025-12-04T09:21:03.9120321Z * [new branch] export-D82250826 -> origin/export-D82250826 2025-12-04T09:21:03.9121924Z * [new branch] export-D82253817 -> origin/export-D82253817 2025-12-04T09:21:03.9123746Z * [new branch] export-D83541846 -> origin/export-D83541846 2025-12-04T09:21:03.9125676Z * [new branch] export-D83627170 -> origin/export-D83627170 2025-12-04T09:21:03.9127580Z * [new branch] export-D83766701 -> origin/export-D83766701 2025-12-04T09:21:03.9129420Z * [new branch] export-D83768878 -> origin/export-D83768878 2025-12-04T09:21:03.9131353Z * [new branch] export-D83769447 -> origin/export-D83769447 2025-12-04T09:21:03.9133166Z * [new branch] export-D84089824 -> origin/export-D84089824 2025-12-04T09:21:03.9134985Z * [new branch] export-D84213020 -> origin/export-D84213020 2025-12-04T09:21:03.9137854Z * [new branch] export-D84373821 -> origin/export-D84373821 2025-12-04T09:21:03.9139780Z * [new branch] export-D84612194 -> origin/export-D84612194 2025-12-04T09:21:03.9141600Z * [new branch] export-D84890985 -> origin/export-D84890985 2025-12-04T09:21:03.9143459Z * [new branch] export-D85122326 -> origin/export-D85122326 2025-12-04T09:21:03.9145661Z * [new branch] export-D86256198 -> origin/export-D86256198 2025-12-04T09:21:03.9147810Z * [new branch] export-D86460608 -> origin/export-D86460608 2025-12-04T09:21:03.9150077Z * [new branch] export-D86474796 -> origin/export-D86474796 2025-12-04T09:21:03.9156833Z * [new branch] export-D86712396 -> origin/export-D86712396 2025-12-04T09:21:03.9158540Z * [new branch] export-D87022129 -> origin/export-D87022129 2025-12-04T09:21:03.9160291Z * [new branch] export-D87838959 -> origin/export-D87838959 2025-12-04T09:21:03.9162039Z * [new branch] export-D88319437 -> origin/export-D88319437 2025-12-04T09:21:03.9164038Z * [new branch] exported-model-train-idempotent -> origin/exported-model-train-idempotent 2025-12-04T09:21:03.9165604Z * [new branch] ezyang-titan-october -> origin/ezyang-titan-october 2025-12-04T09:21:03.9167175Z * [new branch] ezyang-titan-october2 -> origin/ezyang-titan-october2 2025-12-04T09:21:03.9168724Z * [new branch] ezyang-war -> origin/ezyang-war 2025-12-04T09:21:03.9170848Z * [new branch] ezyang/wip-aot-descriptors -> origin/ezyang/wip-aot-descriptors 2025-12-04T09:21:03.9172365Z * [new branch] fa_u8_brgemm -> origin/fa_u8_brgemm 2025-12-04T09:21:03.9174560Z * [new branch] fadeputr/sequence_fbgemm -> origin/fadeputr/sequence_fbgemm 2025-12-04T09:21:03.9176157Z * [new branch] fastmath_baseline -> origin/fastmath_baseline 2025-12-04T09:21:03.9178317Z * [new branch] fbcode/warm -> origin/fbcode/warm 2025-12-04T09:21:03.9180092Z * [new branch] fca -> origin/fca 2025-12-04T09:21:03.9181679Z * [new branch] fca2_ca5984c -> origin/fca2_ca5984c 2025-12-04T09:21:03.9183300Z * [new branch] fca5 -> origin/fca5 2025-12-04T09:21:03.9185426Z * [new branch] feature/justknobs-cpp -> origin/feature/justknobs-cpp 2025-12-04T09:21:03.9187336Z * [new branch] feature/numa-forkserver -> origin/feature/numa-forkserver 2025-12-04T09:21:03.9189587Z * [new branch] ffast_math_baseline -> origin/ffast_math_baseline 2025-12-04T09:21:03.9191201Z * [new branch] ffast_math_target -> origin/ffast_math_target 2025-12-04T09:21:03.9193394Z * [new branch] findhao/base_commit -> origin/findhao/base_commit 2025-12-04T09:21:03.9195048Z * [new branch] findhao/base_commit1 -> origin/findhao/base_commit1 2025-12-04T09:21:03.9196667Z * [new branch] findhao/multistream2 -> origin/findhao/multistream2 2025-12-04T09:21:03.9198245Z * [new branch] findhao/multistream5 -> origin/findhao/multistream5 2025-12-04T09:21:03.9199871Z * [new branch] findhao/multistream6 -> origin/findhao/multistream6 2025-12-04T09:21:03.9202320Z * [new branch] findhao/operatorbench3 -> origin/findhao/operatorbench3 2025-12-04T09:21:03.9204615Z * [new branch] findhao/operatorbench5 -> origin/findhao/operatorbench5 2025-12-04T09:21:03.9206069Z * [new branch] findhao/tritonparse -> origin/findhao/tritonparse 2025-12-04T09:21:03.9207565Z * [new branch] fix-ck-gemm-template-format -> origin/fix-ck-gemm-template-format 2025-12-04T09:21:03.9208934Z * [new branch] fix-config-ignore -> origin/fix-config-ignore 2025-12-04T09:21:03.9210391Z * [new branch] fix-dict-guard -> origin/fix-dict-guard 2025-12-04T09:21:03.9211549Z * [new branch] fix_addmm_issue -> origin/fix_addmm_issue 2025-12-04T09:21:03.9213102Z * [new branch] fix_amd_missing_cluster_dims -> origin/fix_amd_missing_cluster_dims 2025-12-04T09:21:03.9221123Z * [new branch] fix_bench_bwd_pass -> origin/fix_bench_bwd_pass 2025-12-04T09:21:03.9221808Z * [new branch] fix_mem_profiler_config -> origin/fix_mem_profiler_config 2025-12-04T09:21:03.9222506Z * [new branch] fix_nvrtc_discovery -> origin/fix_nvrtc_discovery 2025-12-04T09:21:03.9223130Z * [new branch] fix_op_runner -> origin/fix_op_runner 2025-12-04T09:21:03.9223717Z * [new branch] fix_ubn_159469 -> origin/fix_ubn_159469 2025-12-04T09:21:03.9224836Z * [new branch] fixes-triage -> origin/fixes-triage 2025-12-04T09:21:03.9226026Z * [new branch] fixflashinfer -> origin/fixflashinfer 2025-12-04T09:21:03.9227722Z * [new branch] flash_decoding_cpu -> origin/flash_decoding_cpu 2025-12-04T09:21:03.9228883Z * [new branch] flex-flash -> origin/flex-flash 2025-12-04T09:21:03.9230281Z * [new branch] flex_attention_functorch_grad -> origin/flex_attention_functorch_grad 2025-12-04T09:21:03.9231381Z * [new branch] flex_flash -> origin/flex_flash 2025-12-04T09:21:03.9233396Z * [new branch] fmassa/fix_memeff_sharding_rule -> origin/fmassa/fix_memeff_sharding_rule 2025-12-04T09:21:03.9234631Z * [new branch] fmassa/tests_comm_compute_scheduler -> origin/fmassa/tests_comm_compute_scheduler 2025-12-04T09:21:03.9235729Z * [new branch] forkserver_fix -> origin/forkserver_fix 2025-12-04T09:21:03.9236983Z * [new branch] fsdp2_trace_rules -> origin/fsdp2_trace_rules 2025-12-04T09:21:03.9238466Z * [new branch] fx_cpp -> origin/fx_cpp 2025-12-04T09:21:03.9240196Z * [new branch] fy/fix-win -> origin/fy/fix-win 2025-12-04T09:21:03.9241557Z * [new branch] galv-patch-1 -> origin/galv-patch-1 2025-12-04T09:21:03.9243587Z * [new branch] galv/cudagraphs-conditional-nodes-4 -> origin/galv/cudagraphs-conditional-nodes-4 2025-12-04T09:21:03.9245149Z * [new branch] georgehong/cmakelists-patch -> origin/georgehong/cmakelists-patch 2025-12-04T09:21:03.9247702Z * [new branch] gh/AlnisM/1/base -> origin/gh/AlnisM/1/base 2025-12-04T09:21:03.9248935Z * [new branch] gh/AlnisM/1/head -> origin/gh/AlnisM/1/head 2025-12-04T09:21:03.9251147Z * [new branch] gh/EikanWang/67/base -> origin/gh/EikanWang/67/base 2025-12-04T09:21:03.9252345Z * [new branch] gh/EikanWang/67/head -> origin/gh/EikanWang/67/head 2025-12-04T09:21:03.9254754Z * [new branch] gh/Gasoonjia/1/base -> origin/gh/Gasoonjia/1/base 2025-12-04T09:21:03.9255979Z * [new branch] gh/Gasoonjia/1/head -> origin/gh/Gasoonjia/1/head 2025-12-04T09:21:03.9258023Z * [new branch] gh/H-Huang/131/base -> origin/gh/H-Huang/131/base 2025-12-04T09:21:03.9259196Z * [new branch] gh/H-Huang/131/head -> origin/gh/H-Huang/131/head 2025-12-04T09:21:03.9260386Z * [new branch] gh/H-Huang/131/orig -> origin/gh/H-Huang/131/orig 2025-12-04T09:21:03.9262117Z * [new branch] gh/H-Huang/132/base -> origin/gh/H-Huang/132/base 2025-12-04T09:21:03.9263303Z * [new branch] gh/H-Huang/132/head -> origin/gh/H-Huang/132/head 2025-12-04T09:21:03.9264456Z * [new branch] gh/H-Huang/132/orig -> origin/gh/H-Huang/132/orig 2025-12-04T09:21:03.9266272Z * [new branch] gh/H-Huang/180/base -> origin/gh/H-Huang/180/base 2025-12-04T09:21:03.9267680Z * [new branch] gh/H-Huang/180/head -> origin/gh/H-Huang/180/head 2025-12-04T09:21:03.9268898Z * [new branch] gh/H-Huang/180/orig -> origin/gh/H-Huang/180/orig 2025-12-04T09:21:03.9270561Z * [new branch] gh/H-Huang/182/base -> origin/gh/H-Huang/182/base 2025-12-04T09:21:03.9271740Z * [new branch] gh/H-Huang/182/head -> origin/gh/H-Huang/182/head 2025-12-04T09:21:03.9272978Z * [new branch] gh/H-Huang/182/orig -> origin/gh/H-Huang/182/orig 2025-12-04T09:21:03.9275140Z * [new branch] gh/H-Huang/226/base -> origin/gh/H-Huang/226/base 2025-12-04T09:21:03.9280559Z * [new branch] gh/H-Huang/226/head -> origin/gh/H-Huang/226/head 2025-12-04T09:21:03.9281194Z * [new branch] gh/H-Huang/226/orig -> origin/gh/H-Huang/226/orig 2025-12-04T09:21:03.9281832Z * [new branch] gh/H-Huang/228/base -> origin/gh/H-Huang/228/base 2025-12-04T09:21:03.9282470Z * [new branch] gh/H-Huang/228/head -> origin/gh/H-Huang/228/head 2025-12-04T09:21:03.9283083Z * [new branch] gh/H-Huang/228/orig -> origin/gh/H-Huang/228/orig 2025-12-04T09:21:03.9284146Z * [new branch] gh/IvanKobzarev/150/base -> origin/gh/IvanKobzarev/150/base 2025-12-04T09:21:03.9285326Z * [new branch] gh/IvanKobzarev/150/head -> origin/gh/IvanKobzarev/150/head 2025-12-04T09:21:03.9286564Z * [new branch] gh/IvanKobzarev/150/orig -> origin/gh/IvanKobzarev/150/orig 2025-12-04T09:21:03.9288413Z * [new branch] gh/IvanKobzarev/157/base -> origin/gh/IvanKobzarev/157/base 2025-12-04T09:21:03.9289683Z * [new branch] gh/IvanKobzarev/157/head -> origin/gh/IvanKobzarev/157/head 2025-12-04T09:21:03.9290948Z * [new branch] gh/IvanKobzarev/157/orig -> origin/gh/IvanKobzarev/157/orig 2025-12-04T09:21:03.9292987Z * [new branch] gh/IvanKobzarev/159/base -> origin/gh/IvanKobzarev/159/base 2025-12-04T09:21:03.9294134Z * [new branch] gh/IvanKobzarev/159/head -> origin/gh/IvanKobzarev/159/head 2025-12-04T09:21:03.9295369Z * [new branch] gh/IvanKobzarev/159/orig -> origin/gh/IvanKobzarev/159/orig 2025-12-04T09:21:03.9297116Z * [new branch] gh/IvanKobzarev/162/base -> origin/gh/IvanKobzarev/162/base 2025-12-04T09:21:03.9298421Z * [new branch] gh/IvanKobzarev/162/head -> origin/gh/IvanKobzarev/162/head 2025-12-04T09:21:03.9299593Z * [new branch] gh/IvanKobzarev/162/orig -> origin/gh/IvanKobzarev/162/orig 2025-12-04T09:21:03.9302095Z * [new branch] gh/IvanKobzarev/163/base -> origin/gh/IvanKobzarev/163/base 2025-12-04T09:21:03.9303187Z * [new branch] gh/IvanKobzarev/163/head -> origin/gh/IvanKobzarev/163/head 2025-12-04T09:21:03.9304450Z * [new branch] gh/IvanKobzarev/163/orig -> origin/gh/IvanKobzarev/163/orig 2025-12-04T09:21:03.9306272Z * [new branch] gh/IvanKobzarev/166/base -> origin/gh/IvanKobzarev/166/base 2025-12-04T09:21:03.9307678Z * [new branch] gh/IvanKobzarev/166/head -> origin/gh/IvanKobzarev/166/head 2025-12-04T09:21:03.9308907Z * [new branch] gh/IvanKobzarev/166/orig -> origin/gh/IvanKobzarev/166/orig 2025-12-04T09:21:03.9310751Z * [new branch] gh/IvanKobzarev/167/base -> origin/gh/IvanKobzarev/167/base 2025-12-04T09:21:03.9312466Z * [new branch] gh/IvanKobzarev/167/head -> origin/gh/IvanKobzarev/167/head 2025-12-04T09:21:03.9313537Z * [new branch] gh/IvanKobzarev/167/orig -> origin/gh/IvanKobzarev/167/orig 2025-12-04T09:21:03.9315333Z * [new branch] gh/IvanKobzarev/168/base -> origin/gh/IvanKobzarev/168/base 2025-12-04T09:21:03.9316569Z * [new branch] gh/IvanKobzarev/168/head -> origin/gh/IvanKobzarev/168/head 2025-12-04T09:21:03.9317724Z * [new branch] gh/IvanKobzarev/168/orig -> origin/gh/IvanKobzarev/168/orig 2025-12-04T09:21:03.9319622Z * [new branch] gh/IvanKobzarev/169/base -> origin/gh/IvanKobzarev/169/base 2025-12-04T09:21:03.9320776Z * [new branch] gh/IvanKobzarev/169/head -> origin/gh/IvanKobzarev/169/head 2025-12-04T09:21:03.9321984Z * [new branch] gh/IvanKobzarev/169/orig -> origin/gh/IvanKobzarev/169/orig 2025-12-04T09:21:03.9323520Z * [new branch] gh/IvanKobzarev/170/base -> origin/gh/IvanKobzarev/170/base 2025-12-04T09:21:03.9324688Z * [new branch] gh/IvanKobzarev/170/head -> origin/gh/IvanKobzarev/170/head 2025-12-04T09:21:03.9325914Z * [new branch] gh/IvanKobzarev/170/orig -> origin/gh/IvanKobzarev/170/orig 2025-12-04T09:21:03.9327790Z * [new branch] gh/IvanKobzarev/171/base -> origin/gh/IvanKobzarev/171/base 2025-12-04T09:21:03.9328928Z * [new branch] gh/IvanKobzarev/171/head -> origin/gh/IvanKobzarev/171/head 2025-12-04T09:21:03.9330133Z * [new branch] gh/IvanKobzarev/171/orig -> origin/gh/IvanKobzarev/171/orig 2025-12-04T09:21:03.9331959Z * [new branch] gh/IvanKobzarev/172/base -> origin/gh/IvanKobzarev/172/base 2025-12-04T09:21:03.9333146Z * [new branch] gh/IvanKobzarev/172/head -> origin/gh/IvanKobzarev/172/head 2025-12-04T09:21:03.9334337Z * [new branch] gh/IvanKobzarev/172/orig -> origin/gh/IvanKobzarev/172/orig 2025-12-04T09:21:03.9336017Z * [new branch] gh/IvanKobzarev/173/base -> origin/gh/IvanKobzarev/173/base 2025-12-04T09:21:03.9337170Z * [new branch] gh/IvanKobzarev/173/head -> origin/gh/IvanKobzarev/173/head 2025-12-04T09:21:03.9343012Z * [new branch] gh/IvanKobzarev/173/orig -> origin/gh/IvanKobzarev/173/orig 2025-12-04T09:21:03.9343755Z * [new branch] gh/IvanKobzarev/174/base -> origin/gh/IvanKobzarev/174/base 2025-12-04T09:21:03.9344554Z * [new branch] gh/IvanKobzarev/174/head -> origin/gh/IvanKobzarev/174/head 2025-12-04T09:21:03.9345286Z * [new branch] gh/IvanKobzarev/174/orig -> origin/gh/IvanKobzarev/174/orig 2025-12-04T09:21:03.9345999Z * [new branch] gh/IvanKobzarev/175/base -> origin/gh/IvanKobzarev/175/base 2025-12-04T09:21:03.9346830Z * [new branch] gh/IvanKobzarev/175/head -> origin/gh/IvanKobzarev/175/head 2025-12-04T09:21:03.9348100Z * [new branch] gh/IvanKobzarev/175/orig -> origin/gh/IvanKobzarev/175/orig 2025-12-04T09:21:03.9350012Z * [new branch] gh/IvanKobzarev/176/base -> origin/gh/IvanKobzarev/176/base 2025-12-04T09:21:03.9351191Z * [new branch] gh/IvanKobzarev/176/head -> origin/gh/IvanKobzarev/176/head 2025-12-04T09:21:03.9352406Z * [new branch] gh/IvanKobzarev/176/orig -> origin/gh/IvanKobzarev/176/orig 2025-12-04T09:21:03.9354461Z * [new branch] gh/IvanKobzarev/177/base -> origin/gh/IvanKobzarev/177/base 2025-12-04T09:21:03.9355662Z * [new branch] gh/IvanKobzarev/177/head -> origin/gh/IvanKobzarev/177/head 2025-12-04T09:21:03.9356890Z * [new branch] gh/IvanKobzarev/177/orig -> origin/gh/IvanKobzarev/177/orig 2025-12-04T09:21:03.9358882Z * [new branch] gh/IvanKobzarev/178/base -> origin/gh/IvanKobzarev/178/base 2025-12-04T09:21:03.9360076Z * [new branch] gh/IvanKobzarev/178/head -> origin/gh/IvanKobzarev/178/head 2025-12-04T09:21:03.9361301Z * [new branch] gh/IvanKobzarev/178/orig -> origin/gh/IvanKobzarev/178/orig 2025-12-04T09:21:03.9363550Z * [new branch] gh/IvanKobzarev/179/base -> origin/gh/IvanKobzarev/179/base 2025-12-04T09:21:03.9364651Z * [new branch] gh/IvanKobzarev/179/head -> origin/gh/IvanKobzarev/179/head 2025-12-04T09:21:03.9365961Z * [new branch] gh/IvanKobzarev/179/orig -> origin/gh/IvanKobzarev/179/orig 2025-12-04T09:21:03.9367663Z * [new branch] gh/IvanKobzarev/180/base -> origin/gh/IvanKobzarev/180/base 2025-12-04T09:21:03.9368829Z * [new branch] gh/IvanKobzarev/180/head -> origin/gh/IvanKobzarev/180/head 2025-12-04T09:21:03.9370067Z * [new branch] gh/IvanKobzarev/180/orig -> origin/gh/IvanKobzarev/180/orig 2025-12-04T09:21:03.9372018Z * [new branch] gh/IvanKobzarev/181/base -> origin/gh/IvanKobzarev/181/base 2025-12-04T09:21:03.9373227Z * [new branch] gh/IvanKobzarev/181/head -> origin/gh/IvanKobzarev/181/head 2025-12-04T09:21:03.9374424Z * [new branch] gh/IvanKobzarev/181/orig -> origin/gh/IvanKobzarev/181/orig 2025-12-04T09:21:03.9376481Z * [new branch] gh/IvanKobzarev/182/base -> origin/gh/IvanKobzarev/182/base 2025-12-04T09:21:03.9377546Z * [new branch] gh/IvanKobzarev/182/head -> origin/gh/IvanKobzarev/182/head 2025-12-04T09:21:03.9378699Z * [new branch] gh/IvanKobzarev/182/orig -> origin/gh/IvanKobzarev/182/orig 2025-12-04T09:21:03.9380574Z * [new branch] gh/IvanKobzarev/183/base -> origin/gh/IvanKobzarev/183/base 2025-12-04T09:21:03.9381760Z * [new branch] gh/IvanKobzarev/183/head -> origin/gh/IvanKobzarev/183/head 2025-12-04T09:21:03.9383001Z * [new branch] gh/IvanKobzarev/183/orig -> origin/gh/IvanKobzarev/183/orig 2025-12-04T09:21:03.9384722Z * [new branch] gh/IvanKobzarev/184/base -> origin/gh/IvanKobzarev/184/base 2025-12-04T09:21:03.9385923Z * [new branch] gh/IvanKobzarev/184/head -> origin/gh/IvanKobzarev/184/head 2025-12-04T09:21:03.9387510Z * [new branch] gh/IvanKobzarev/184/orig -> origin/gh/IvanKobzarev/184/orig 2025-12-04T09:21:03.9389652Z * [new branch] gh/NikhilAPatel/1/base -> origin/gh/NikhilAPatel/1/base 2025-12-04T09:21:03.9390994Z * [new branch] gh/NikhilAPatel/1/head -> origin/gh/NikhilAPatel/1/head 2025-12-04T09:21:03.9392540Z * [new branch] gh/NikhilAPatel/2/base -> origin/gh/NikhilAPatel/2/base 2025-12-04T09:21:03.9393678Z * [new branch] gh/NikhilAPatel/2/head -> origin/gh/NikhilAPatel/2/head 2025-12-04T09:21:03.9395625Z * [new branch] gh/NikhilAPatel/4/base -> origin/gh/NikhilAPatel/4/base 2025-12-04T09:21:03.9396910Z * [new branch] gh/NikhilAPatel/4/head -> origin/gh/NikhilAPatel/4/head 2025-12-04T09:21:03.9398605Z * [new branch] gh/NikhilAPatel/5/base -> origin/gh/NikhilAPatel/5/base 2025-12-04T09:21:03.9399781Z * [new branch] gh/NikhilAPatel/5/head -> origin/gh/NikhilAPatel/5/head 2025-12-04T09:21:03.9407028Z * [new branch] gh/NikhilAPatel/5/orig -> origin/gh/NikhilAPatel/5/orig 2025-12-04T09:21:03.9408966Z * [new branch] gh/PaliC/17/base -> origin/gh/PaliC/17/base 2025-12-04T09:21:03.9410223Z * [new branch] gh/PaliC/17/head -> origin/gh/PaliC/17/head 2025-12-04T09:21:03.9411470Z * [new branch] gh/PaliC/17/orig -> origin/gh/PaliC/17/orig 2025-12-04T09:21:03.9413302Z * [new branch] gh/PaliC/18/base -> origin/gh/PaliC/18/base 2025-12-04T09:21:03.9414492Z * [new branch] gh/PaliC/18/head -> origin/gh/PaliC/18/head 2025-12-04T09:21:03.9415662Z * [new branch] gh/PaliC/18/orig -> origin/gh/PaliC/18/orig 2025-12-04T09:21:03.9417397Z * [new branch] gh/PaliC/20/base -> origin/gh/PaliC/20/base 2025-12-04T09:21:03.9418513Z * [new branch] gh/PaliC/20/head -> origin/gh/PaliC/20/head 2025-12-04T09:21:03.9419912Z * [new branch] gh/PaliC/20/orig -> origin/gh/PaliC/20/orig 2025-12-04T09:21:03.9421341Z * [new branch] gh/PaliC/21/base -> origin/gh/PaliC/21/base 2025-12-04T09:21:03.9422600Z * [new branch] gh/PaliC/21/head -> origin/gh/PaliC/21/head 2025-12-04T09:21:03.9423710Z * [new branch] gh/PaliC/21/orig -> origin/gh/PaliC/21/orig 2025-12-04T09:21:03.9425635Z * [new branch] gh/PaliC/23/base -> origin/gh/PaliC/23/base 2025-12-04T09:21:03.9426886Z * [new branch] gh/PaliC/23/head -> origin/gh/PaliC/23/head 2025-12-04T09:21:03.9428426Z * [new branch] gh/PaliC/23/orig -> origin/gh/PaliC/23/orig 2025-12-04T09:21:03.9430185Z * [new branch] gh/PaliC/24/base -> origin/gh/PaliC/24/base 2025-12-04T09:21:03.9431314Z * [new branch] gh/PaliC/24/head -> origin/gh/PaliC/24/head 2025-12-04T09:21:03.9432546Z * [new branch] gh/PaliC/24/orig -> origin/gh/PaliC/24/orig 2025-12-04T09:21:03.9434253Z * [new branch] gh/PaliC/25/head -> origin/gh/PaliC/25/head 2025-12-04T09:21:03.9435391Z * [new branch] gh/PaliC/25/next -> origin/gh/PaliC/25/next 2025-12-04T09:21:03.9436651Z * [new branch] gh/PaliC/25/orig -> origin/gh/PaliC/25/orig 2025-12-04T09:21:03.9438361Z * [new branch] gh/PaliC/26/head -> origin/gh/PaliC/26/head 2025-12-04T09:21:03.9439510Z * [new branch] gh/PaliC/26/next -> origin/gh/PaliC/26/next 2025-12-04T09:21:03.9440719Z * [new branch] gh/PaliC/26/orig -> origin/gh/PaliC/26/orig 2025-12-04T09:21:03.9442347Z * [new branch] gh/PaliC/27/next -> origin/gh/PaliC/27/next 2025-12-04T09:21:03.9443914Z * [new branch] gh/PaliC/28/head -> origin/gh/PaliC/28/head 2025-12-04T09:21:03.9444900Z * [new branch] gh/PaliC/28/next -> origin/gh/PaliC/28/next 2025-12-04T09:21:03.9446125Z * [new branch] gh/PaliC/28/orig -> origin/gh/PaliC/28/orig 2025-12-04T09:21:03.9447788Z * [new branch] gh/PaliC/29/head -> origin/gh/PaliC/29/head 2025-12-04T09:21:03.9448835Z * [new branch] gh/PaliC/29/next -> origin/gh/PaliC/29/next 2025-12-04T09:21:03.9449998Z * [new branch] gh/PaliC/29/orig -> origin/gh/PaliC/29/orig 2025-12-04T09:21:03.9452036Z * [new branch] gh/PaliC/30/head -> origin/gh/PaliC/30/head 2025-12-04T09:21:03.9453041Z * [new branch] gh/PaliC/30/next -> origin/gh/PaliC/30/next 2025-12-04T09:21:03.9454259Z * [new branch] gh/PaliC/30/orig -> origin/gh/PaliC/30/orig 2025-12-04T09:21:03.9455874Z * [new branch] gh/PaliC/31/head -> origin/gh/PaliC/31/head 2025-12-04T09:21:03.9456840Z * [new branch] gh/PaliC/31/next -> origin/gh/PaliC/31/next 2025-12-04T09:21:03.9458023Z * [new branch] gh/PaliC/31/orig -> origin/gh/PaliC/31/orig 2025-12-04T09:21:03.9459977Z * [new branch] gh/PaulZhang12/25/base -> origin/gh/PaulZhang12/25/base 2025-12-04T09:21:03.9461214Z * [new branch] gh/PaulZhang12/25/head -> origin/gh/PaulZhang12/25/head 2025-12-04T09:21:03.9462390Z * [new branch] gh/PaulZhang12/25/orig -> origin/gh/PaulZhang12/25/orig 2025-12-04T09:21:03.9470843Z * [new branch] gh/PaulZhang12/28/base -> origin/gh/PaulZhang12/28/base 2025-12-04T09:21:03.9471555Z * [new branch] gh/PaulZhang12/28/head -> origin/gh/PaulZhang12/28/head 2025-12-04T09:21:03.9472360Z * [new branch] gh/PaulZhang12/28/orig -> origin/gh/PaulZhang12/28/orig 2025-12-04T09:21:03.9473929Z * [new branch] gh/PaulZhang12/31/base -> origin/gh/PaulZhang12/31/base 2025-12-04T09:21:03.9475020Z * [new branch] gh/PaulZhang12/31/head -> origin/gh/PaulZhang12/31/head 2025-12-04T09:21:03.9476345Z * [new branch] gh/PaulZhang12/31/orig -> origin/gh/PaulZhang12/31/orig 2025-12-04T09:21:03.9478133Z * [new branch] gh/PaulZhang12/37/base -> origin/gh/PaulZhang12/37/base 2025-12-04T09:21:03.9479329Z * [new branch] gh/PaulZhang12/37/head -> origin/gh/PaulZhang12/37/head 2025-12-04T09:21:03.9480561Z * [new branch] gh/PaulZhang12/37/orig -> origin/gh/PaulZhang12/37/orig 2025-12-04T09:21:03.9482216Z * [new branch] gh/PaulZhang12/40/base -> origin/gh/PaulZhang12/40/base 2025-12-04T09:21:03.9483339Z * [new branch] gh/PaulZhang12/40/head -> origin/gh/PaulZhang12/40/head 2025-12-04T09:21:03.9484583Z * [new branch] gh/PaulZhang12/40/orig -> origin/gh/PaulZhang12/40/orig 2025-12-04T09:21:03.9486300Z * [new branch] gh/PaulZhang12/42/base -> origin/gh/PaulZhang12/42/base 2025-12-04T09:21:03.9487533Z * [new branch] gh/PaulZhang12/42/head -> origin/gh/PaulZhang12/42/head 2025-12-04T09:21:03.9490342Z * [new branch] gh/PaulZhang12/43/base -> origin/gh/PaulZhang12/43/base 2025-12-04T09:21:03.9491300Z * [new branch] gh/PaulZhang12/43/head -> origin/gh/PaulZhang12/43/head 2025-12-04T09:21:03.9492324Z * [new branch] gh/PaulZhang12/43/orig -> origin/gh/PaulZhang12/43/orig 2025-12-04T09:21:03.9493041Z * [new branch] gh/PaulZhang12/44/base -> origin/gh/PaulZhang12/44/base 2025-12-04T09:21:03.9494223Z * [new branch] gh/PaulZhang12/44/head -> origin/gh/PaulZhang12/44/head 2025-12-04T09:21:03.9496014Z * [new branch] gh/PaulZhang12/45/base -> origin/gh/PaulZhang12/45/base 2025-12-04T09:21:03.9497077Z * [new branch] gh/PaulZhang12/45/head -> origin/gh/PaulZhang12/45/head 2025-12-04T09:21:03.9498216Z * [new branch] gh/PaulZhang12/45/orig -> origin/gh/PaulZhang12/45/orig 2025-12-04T09:21:03.9499973Z * [new branch] gh/PaulZhang12/46/base -> origin/gh/PaulZhang12/46/base 2025-12-04T09:21:03.9501834Z * [new branch] gh/PaulZhang12/46/head -> origin/gh/PaulZhang12/46/head 2025-12-04T09:21:03.9503293Z * [new branch] gh/PaulZhang12/46/orig -> origin/gh/PaulZhang12/46/orig 2025-12-04T09:21:03.9505080Z * [new branch] gh/PaulZhang12/47/base -> origin/gh/PaulZhang12/47/base 2025-12-04T09:21:03.9506327Z * [new branch] gh/PaulZhang12/47/head -> origin/gh/PaulZhang12/47/head 2025-12-04T09:21:03.9507898Z * [new branch] gh/PaulZhang12/47/orig -> origin/gh/PaulZhang12/47/orig 2025-12-04T09:21:03.9509424Z * [new branch] gh/PaulZhang12/48/base -> origin/gh/PaulZhang12/48/base 2025-12-04T09:21:03.9510594Z * [new branch] gh/PaulZhang12/48/head -> origin/gh/PaulZhang12/48/head 2025-12-04T09:21:03.9511814Z * [new branch] gh/PaulZhang12/48/orig -> origin/gh/PaulZhang12/48/orig 2025-12-04T09:21:03.9513908Z * [new branch] gh/SamGinzburg/11/base -> origin/gh/SamGinzburg/11/base 2025-12-04T09:21:03.9515138Z * [new branch] gh/SamGinzburg/11/head -> origin/gh/SamGinzburg/11/head 2025-12-04T09:21:03.9517377Z * [new branch] gh/SherlockNoMad/1/base -> origin/gh/SherlockNoMad/1/base 2025-12-04T09:21:03.9518746Z * [new branch] gh/SherlockNoMad/1/head -> origin/gh/SherlockNoMad/1/head 2025-12-04T09:21:03.9520441Z * [new branch] gh/SherlockNoMad/10/base -> origin/gh/SherlockNoMad/10/base 2025-12-04T09:21:03.9521667Z * [new branch] gh/SherlockNoMad/10/head -> origin/gh/SherlockNoMad/10/head 2025-12-04T09:21:03.9522971Z * [new branch] gh/SherlockNoMad/10/orig -> origin/gh/SherlockNoMad/10/orig 2025-12-04T09:21:03.9524501Z * [new branch] gh/SherlockNoMad/11/base -> origin/gh/SherlockNoMad/11/base 2025-12-04T09:21:03.9530590Z * [new branch] gh/SherlockNoMad/11/head -> origin/gh/SherlockNoMad/11/head 2025-12-04T09:21:03.9531446Z * [new branch] gh/SherlockNoMad/11/orig -> origin/gh/SherlockNoMad/11/orig 2025-12-04T09:21:03.9532176Z * [new branch] gh/SherlockNoMad/12/base -> origin/gh/SherlockNoMad/12/base 2025-12-04T09:21:03.9532888Z * [new branch] gh/SherlockNoMad/12/head -> origin/gh/SherlockNoMad/12/head 2025-12-04T09:21:03.9533615Z * [new branch] gh/SherlockNoMad/12/orig -> origin/gh/SherlockNoMad/12/orig 2025-12-04T09:21:03.9534342Z * [new branch] gh/SherlockNoMad/15/base -> origin/gh/SherlockNoMad/15/base 2025-12-04T09:21:03.9535082Z * [new branch] gh/SherlockNoMad/15/head -> origin/gh/SherlockNoMad/15/head 2025-12-04T09:21:03.9535948Z * [new branch] gh/SherlockNoMad/15/orig -> origin/gh/SherlockNoMad/15/orig 2025-12-04T09:21:03.9537641Z * [new branch] gh/SherlockNoMad/17/base -> origin/gh/SherlockNoMad/17/base 2025-12-04T09:21:03.9538910Z * [new branch] gh/SherlockNoMad/17/head -> origin/gh/SherlockNoMad/17/head 2025-12-04T09:21:03.9540136Z * [new branch] gh/SherlockNoMad/17/orig -> origin/gh/SherlockNoMad/17/orig 2025-12-04T09:21:03.9541949Z * [new branch] gh/SherlockNoMad/18/base -> origin/gh/SherlockNoMad/18/base 2025-12-04T09:21:03.9543160Z * [new branch] gh/SherlockNoMad/18/head -> origin/gh/SherlockNoMad/18/head 2025-12-04T09:21:03.9544363Z * [new branch] gh/SherlockNoMad/18/orig -> origin/gh/SherlockNoMad/18/orig 2025-12-04T09:21:03.9545906Z * [new branch] gh/SherlockNoMad/19/base -> origin/gh/SherlockNoMad/19/base 2025-12-04T09:21:03.9547459Z * [new branch] gh/SherlockNoMad/19/head -> origin/gh/SherlockNoMad/19/head 2025-12-04T09:21:03.9548746Z * [new branch] gh/SherlockNoMad/19/orig -> origin/gh/SherlockNoMad/19/orig 2025-12-04T09:21:03.9550416Z * [new branch] gh/SherlockNoMad/2/base -> origin/gh/SherlockNoMad/2/base 2025-12-04T09:21:03.9551513Z * [new branch] gh/SherlockNoMad/2/head -> origin/gh/SherlockNoMad/2/head 2025-12-04T09:21:03.9553092Z * [new branch] gh/SherlockNoMad/20/base -> origin/gh/SherlockNoMad/20/base 2025-12-04T09:21:03.9554298Z * [new branch] gh/SherlockNoMad/20/head -> origin/gh/SherlockNoMad/20/head 2025-12-04T09:21:03.9555410Z * [new branch] gh/SherlockNoMad/20/orig -> origin/gh/SherlockNoMad/20/orig 2025-12-04T09:21:03.9557377Z * [new branch] gh/SherlockNoMad/21/base -> origin/gh/SherlockNoMad/21/base 2025-12-04T09:21:03.9558614Z * [new branch] gh/SherlockNoMad/21/head -> origin/gh/SherlockNoMad/21/head 2025-12-04T09:21:03.9559879Z * [new branch] gh/SherlockNoMad/21/orig -> origin/gh/SherlockNoMad/21/orig 2025-12-04T09:21:03.9561411Z * [new branch] gh/SherlockNoMad/3/base -> origin/gh/SherlockNoMad/3/base 2025-12-04T09:21:03.9562556Z * [new branch] gh/SherlockNoMad/3/head -> origin/gh/SherlockNoMad/3/head 2025-12-04T09:21:03.9564001Z * [new branch] gh/SherlockNoMad/4/base -> origin/gh/SherlockNoMad/4/base 2025-12-04T09:21:03.9565092Z * [new branch] gh/SherlockNoMad/4/head -> origin/gh/SherlockNoMad/4/head 2025-12-04T09:21:03.9566650Z * [new branch] gh/SherlockNoMad/5/base -> origin/gh/SherlockNoMad/5/base 2025-12-04T09:21:03.9567749Z * [new branch] gh/SherlockNoMad/5/head -> origin/gh/SherlockNoMad/5/head 2025-12-04T09:21:03.9570178Z * [new branch] gh/Sidharth123-cpu/24/base -> origin/gh/Sidharth123-cpu/24/base 2025-12-04T09:21:03.9571592Z * [new branch] gh/Sidharth123-cpu/25/base -> origin/gh/Sidharth123-cpu/25/base 2025-12-04T09:21:03.9573145Z * [new branch] gh/Sidharth123-cpu/26/base -> origin/gh/Sidharth123-cpu/26/base 2025-12-04T09:21:03.9574824Z * [new branch] gh/Sidharth123-cpu/27/base -> origin/gh/Sidharth123-cpu/27/base 2025-12-04T09:21:03.9576822Z * [new branch] gh/StrongerXi/1/base -> origin/gh/StrongerXi/1/base 2025-12-04T09:21:03.9577923Z * [new branch] gh/StrongerXi/1/head -> origin/gh/StrongerXi/1/head 2025-12-04T09:21:03.9579675Z * [new branch] gh/StrongerXi/71/base -> origin/gh/StrongerXi/71/base 2025-12-04T09:21:03.9580812Z * [new branch] gh/StrongerXi/71/head -> origin/gh/StrongerXi/71/head 2025-12-04T09:21:03.9582453Z * [new branch] gh/StrongerXi/72/base -> origin/gh/StrongerXi/72/base 2025-12-04T09:21:03.9583520Z * [new branch] gh/StrongerXi/72/head -> origin/gh/StrongerXi/72/head 2025-12-04T09:21:03.9585169Z * [new branch] gh/StrongerXi/73/base -> origin/gh/StrongerXi/73/base 2025-12-04T09:21:03.9586284Z * [new branch] gh/StrongerXi/73/head -> origin/gh/StrongerXi/73/head 2025-12-04T09:21:03.9593047Z * [new branch] gh/StrongerXi/73/orig -> origin/gh/StrongerXi/73/orig 2025-12-04T09:21:03.9593723Z * [new branch] gh/XilunWu/160/base -> origin/gh/XilunWu/160/base 2025-12-04T09:21:03.9594381Z * [new branch] gh/XilunWu/160/head -> origin/gh/XilunWu/160/head 2025-12-04T09:21:03.9595017Z * [new branch] gh/XilunWu/160/orig -> origin/gh/XilunWu/160/orig 2025-12-04T09:21:03.9595669Z * [new branch] gh/XilunWu/163/base -> origin/gh/XilunWu/163/base 2025-12-04T09:21:03.9596324Z * [new branch] gh/XilunWu/163/head -> origin/gh/XilunWu/163/head 2025-12-04T09:21:03.9597180Z * [new branch] gh/XilunWu/163/orig -> origin/gh/XilunWu/163/orig 2025-12-04T09:21:03.9599080Z * [new branch] gh/XilunWu/168/base -> origin/gh/XilunWu/168/base 2025-12-04T09:21:03.9600150Z * [new branch] gh/XilunWu/168/head -> origin/gh/XilunWu/168/head 2025-12-04T09:21:03.9602092Z * [new branch] gh/XilunWu/168/orig -> origin/gh/XilunWu/168/orig 2025-12-04T09:21:03.9603817Z * [new branch] gh/XilunWu/169/base -> origin/gh/XilunWu/169/base 2025-12-04T09:21:03.9605001Z * [new branch] gh/XilunWu/169/head -> origin/gh/XilunWu/169/head 2025-12-04T09:21:03.9606224Z * [new branch] gh/XilunWu/169/orig -> origin/gh/XilunWu/169/orig 2025-12-04T09:21:03.9607832Z * [new branch] gh/XilunWu/170/base -> origin/gh/XilunWu/170/base 2025-12-04T09:21:03.9609034Z * [new branch] gh/XilunWu/170/head -> origin/gh/XilunWu/170/head 2025-12-04T09:21:03.9610283Z * [new branch] gh/XilunWu/170/orig -> origin/gh/XilunWu/170/orig 2025-12-04T09:21:03.9612239Z * [new branch] gh/XilunWu/171/base -> origin/gh/XilunWu/171/base 2025-12-04T09:21:03.9613521Z * [new branch] gh/XilunWu/171/head -> origin/gh/XilunWu/171/head 2025-12-04T09:21:03.9614734Z * [new branch] gh/XilunWu/171/orig -> origin/gh/XilunWu/171/orig 2025-12-04T09:21:03.9616315Z * [new branch] gh/XilunWu/173/base -> origin/gh/XilunWu/173/base 2025-12-04T09:21:03.9617428Z * [new branch] gh/XilunWu/173/head -> origin/gh/XilunWu/173/head 2025-12-04T09:21:03.9618611Z * [new branch] gh/XilunWu/173/orig -> origin/gh/XilunWu/173/orig 2025-12-04T09:21:03.9620224Z * [new branch] gh/XilunWu/175/base -> origin/gh/XilunWu/175/base 2025-12-04T09:21:03.9621448Z * [new branch] gh/XilunWu/175/head -> origin/gh/XilunWu/175/head 2025-12-04T09:21:03.9622669Z * [new branch] gh/XilunWu/175/orig -> origin/gh/XilunWu/175/orig 2025-12-04T09:21:03.9624350Z * [new branch] gh/XilunWu/176/base -> origin/gh/XilunWu/176/base 2025-12-04T09:21:03.9625571Z * [new branch] gh/XilunWu/176/head -> origin/gh/XilunWu/176/head 2025-12-04T09:21:03.9626954Z * [new branch] gh/XilunWu/176/orig -> origin/gh/XilunWu/176/orig 2025-12-04T09:21:03.9629339Z * [new branch] gh/XuehaiPan/14/base -> origin/gh/XuehaiPan/14/base 2025-12-04T09:21:03.9630519Z * [new branch] gh/XuehaiPan/14/head -> origin/gh/XuehaiPan/14/head 2025-12-04T09:21:03.9631750Z * [new branch] gh/XuehaiPan/14/orig -> origin/gh/XuehaiPan/14/orig 2025-12-04T09:21:03.9633499Z * [new branch] gh/XuehaiPan/179/base -> origin/gh/XuehaiPan/179/base 2025-12-04T09:21:03.9634667Z * [new branch] gh/XuehaiPan/179/head -> origin/gh/XuehaiPan/179/head 2025-12-04T09:21:03.9636076Z * [new branch] gh/XuehaiPan/179/orig -> origin/gh/XuehaiPan/179/orig 2025-12-04T09:21:03.9637610Z * [new branch] gh/XuehaiPan/249/base -> origin/gh/XuehaiPan/249/base 2025-12-04T09:21:03.9638794Z * [new branch] gh/XuehaiPan/249/head -> origin/gh/XuehaiPan/249/head 2025-12-04T09:21:03.9640288Z * [new branch] gh/XuehaiPan/249/orig -> origin/gh/XuehaiPan/249/orig 2025-12-04T09:21:03.9642179Z * [new branch] gh/XuehaiPan/253/base -> origin/gh/XuehaiPan/253/base 2025-12-04T09:21:03.9643338Z * [new branch] gh/XuehaiPan/253/head -> origin/gh/XuehaiPan/253/head 2025-12-04T09:21:03.9644514Z * [new branch] gh/XuehaiPan/253/orig -> origin/gh/XuehaiPan/253/orig 2025-12-04T09:21:03.9646347Z * [new branch] gh/XuehaiPan/254/base -> origin/gh/XuehaiPan/254/base 2025-12-04T09:21:03.9647457Z * [new branch] gh/XuehaiPan/254/head -> origin/gh/XuehaiPan/254/head 2025-12-04T09:21:03.9648648Z * [new branch] gh/XuehaiPan/254/orig -> origin/gh/XuehaiPan/254/orig 2025-12-04T09:21:03.9650253Z * [new branch] gh/XuehaiPan/255/base -> origin/gh/XuehaiPan/255/base 2025-12-04T09:21:03.9657043Z * [new branch] gh/XuehaiPan/255/head -> origin/gh/XuehaiPan/255/head 2025-12-04T09:21:03.9658052Z * [new branch] gh/XuehaiPan/255/orig -> origin/gh/XuehaiPan/255/orig 2025-12-04T09:21:03.9659727Z * [new branch] gh/XuehaiPan/271/base -> origin/gh/XuehaiPan/271/base 2025-12-04T09:21:03.9661192Z * [new branch] gh/XuehaiPan/271/head -> origin/gh/XuehaiPan/271/head 2025-12-04T09:21:03.9662460Z * [new branch] gh/XuehaiPan/271/orig -> origin/gh/XuehaiPan/271/orig 2025-12-04T09:21:03.9664287Z * [new branch] gh/XuehaiPan/343/base -> origin/gh/XuehaiPan/343/base 2025-12-04T09:21:03.9665456Z * [new branch] gh/XuehaiPan/343/head -> origin/gh/XuehaiPan/343/head 2025-12-04T09:21:03.9666718Z * [new branch] gh/XuehaiPan/343/orig -> origin/gh/XuehaiPan/343/orig 2025-12-04T09:21:03.9668831Z * [new branch] gh/XuehaiPan/347/base -> origin/gh/XuehaiPan/347/base 2025-12-04T09:21:03.9670019Z * [new branch] gh/XuehaiPan/347/head -> origin/gh/XuehaiPan/347/head 2025-12-04T09:21:03.9671263Z * [new branch] gh/XuehaiPan/347/orig -> origin/gh/XuehaiPan/347/orig 2025-12-04T09:21:03.9672943Z * [new branch] gh/XuehaiPan/348/base -> origin/gh/XuehaiPan/348/base 2025-12-04T09:21:03.9674139Z * [new branch] gh/XuehaiPan/348/head -> origin/gh/XuehaiPan/348/head 2025-12-04T09:21:03.9675400Z * [new branch] gh/XuehaiPan/348/orig -> origin/gh/XuehaiPan/348/orig 2025-12-04T09:21:03.9677079Z * [new branch] gh/XuehaiPan/350/base -> origin/gh/XuehaiPan/350/base 2025-12-04T09:21:03.9678389Z * [new branch] gh/XuehaiPan/350/head -> origin/gh/XuehaiPan/350/head 2025-12-04T09:21:03.9679558Z * [new branch] gh/XuehaiPan/350/orig -> origin/gh/XuehaiPan/350/orig 2025-12-04T09:21:03.9681318Z * [new branch] gh/XuehaiPan/365/base -> origin/gh/XuehaiPan/365/base 2025-12-04T09:21:03.9682445Z * [new branch] gh/XuehaiPan/365/head -> origin/gh/XuehaiPan/365/head 2025-12-04T09:21:03.9683605Z * [new branch] gh/XuehaiPan/365/orig -> origin/gh/XuehaiPan/365/orig 2025-12-04T09:21:03.9685372Z * [new branch] gh/XuehaiPan/366/base -> origin/gh/XuehaiPan/366/base 2025-12-04T09:21:03.9686518Z * [new branch] gh/XuehaiPan/366/head -> origin/gh/XuehaiPan/366/head 2025-12-04T09:21:03.9688194Z * [new branch] gh/XuehaiPan/370/base -> origin/gh/XuehaiPan/370/base 2025-12-04T09:21:03.9689359Z * [new branch] gh/XuehaiPan/370/head -> origin/gh/XuehaiPan/370/head 2025-12-04T09:21:03.9690527Z * [new branch] gh/XuehaiPan/370/orig -> origin/gh/XuehaiPan/370/orig 2025-12-04T09:21:03.9692204Z * [new branch] gh/XuehaiPan/390/base -> origin/gh/XuehaiPan/390/base 2025-12-04T09:21:03.9693353Z * [new branch] gh/XuehaiPan/390/head -> origin/gh/XuehaiPan/390/head 2025-12-04T09:21:03.9694558Z * [new branch] gh/XuehaiPan/390/orig -> origin/gh/XuehaiPan/390/orig 2025-12-04T09:21:03.9696162Z * [new branch] gh/XuehaiPan/391/base -> origin/gh/XuehaiPan/391/base 2025-12-04T09:21:03.9697294Z * [new branch] gh/XuehaiPan/391/head -> origin/gh/XuehaiPan/391/head 2025-12-04T09:21:03.9698515Z * [new branch] gh/XuehaiPan/391/orig -> origin/gh/XuehaiPan/391/orig 2025-12-04T09:21:03.9700268Z * [new branch] gh/XuehaiPan/392/base -> origin/gh/XuehaiPan/392/base 2025-12-04T09:21:03.9702480Z * [new branch] gh/XuehaiPan/392/head -> origin/gh/XuehaiPan/392/head 2025-12-04T09:21:03.9703660Z * [new branch] gh/XuehaiPan/392/orig -> origin/gh/XuehaiPan/392/orig 2025-12-04T09:21:03.9705913Z * [new branch] gh/XuehaiPan/394/base -> origin/gh/XuehaiPan/394/base 2025-12-04T09:21:03.9707230Z * [new branch] gh/XuehaiPan/394/head -> origin/gh/XuehaiPan/394/head 2025-12-04T09:21:03.9708501Z * [new branch] gh/XuehaiPan/394/orig -> origin/gh/XuehaiPan/394/orig 2025-12-04T09:21:03.9710254Z * [new branch] gh/XuehaiPan/397/base -> origin/gh/XuehaiPan/397/base 2025-12-04T09:21:03.9711482Z * [new branch] gh/XuehaiPan/397/head -> origin/gh/XuehaiPan/397/head 2025-12-04T09:21:03.9712769Z * [new branch] gh/XuehaiPan/397/orig -> origin/gh/XuehaiPan/397/orig 2025-12-04T09:21:03.9719600Z * [new branch] gh/XuehaiPan/398/base -> origin/gh/XuehaiPan/398/base 2025-12-04T09:21:03.9720767Z * [new branch] gh/XuehaiPan/398/head -> origin/gh/XuehaiPan/398/head 2025-12-04T09:21:03.9721971Z * [new branch] gh/XuehaiPan/398/orig -> origin/gh/XuehaiPan/398/orig 2025-12-04T09:21:03.9723731Z * [new branch] gh/XuehaiPan/399/base -> origin/gh/XuehaiPan/399/base 2025-12-04T09:21:03.9725023Z * [new branch] gh/XuehaiPan/399/head -> origin/gh/XuehaiPan/399/head 2025-12-04T09:21:03.9726235Z * [new branch] gh/XuehaiPan/399/orig -> origin/gh/XuehaiPan/399/orig 2025-12-04T09:21:03.9727908Z * [new branch] gh/XuehaiPan/400/base -> origin/gh/XuehaiPan/400/base 2025-12-04T09:21:03.9729046Z * [new branch] gh/XuehaiPan/400/head -> origin/gh/XuehaiPan/400/head 2025-12-04T09:21:03.9730232Z * [new branch] gh/XuehaiPan/400/orig -> origin/gh/XuehaiPan/400/orig 2025-12-04T09:21:03.9759628Z * [new branch] gh/ZhiweiYan-96/39/base -> origin/gh/ZhiweiYan-96/39/base 2025-12-04T09:21:03.9760625Z * [new branch] gh/ZhiweiYan-96/39/head -> origin/gh/ZhiweiYan-96/39/head 2025-12-04T09:21:03.9761488Z * [new branch] gh/ZhiweiYan-96/39/orig -> origin/gh/ZhiweiYan-96/39/orig 2025-12-04T09:21:03.9762433Z * [new branch] gh/ZhiweiYan-96/44/base -> origin/gh/ZhiweiYan-96/44/base 2025-12-04T09:21:03.9763181Z * [new branch] gh/ZhiweiYan-96/44/head -> origin/gh/ZhiweiYan-96/44/head 2025-12-04T09:21:03.9763838Z * [new branch] gh/ZhiweiYan-96/45/base -> origin/gh/ZhiweiYan-96/45/base 2025-12-04T09:21:03.9764540Z * [new branch] gh/ZhiweiYan-96/45/head -> origin/gh/ZhiweiYan-96/45/head 2025-12-04T09:21:03.9765209Z * [new branch] gh/ZhiweiYan-96/49/base -> origin/gh/ZhiweiYan-96/49/base 2025-12-04T09:21:03.9765940Z * [new branch] gh/ZhiweiYan-96/49/head -> origin/gh/ZhiweiYan-96/49/head 2025-12-04T09:21:03.9766603Z * [new branch] gh/ZhiweiYan-96/62/base -> origin/gh/ZhiweiYan-96/62/base 2025-12-04T09:21:03.9767332Z * [new branch] gh/ZhiweiYan-96/62/head -> origin/gh/ZhiweiYan-96/62/head 2025-12-04T09:21:03.9768001Z * [new branch] gh/ZhiweiYan-96/66/base -> origin/gh/ZhiweiYan-96/66/base 2025-12-04T09:21:03.9768736Z * [new branch] gh/ZhiweiYan-96/66/head -> origin/gh/ZhiweiYan-96/66/head 2025-12-04T09:21:03.9769395Z * [new branch] gh/ZhiweiYan-96/67/base -> origin/gh/ZhiweiYan-96/67/base 2025-12-04T09:21:03.9770119Z * [new branch] gh/ZhiweiYan-96/67/head -> origin/gh/ZhiweiYan-96/67/head 2025-12-04T09:21:03.9770791Z * [new branch] gh/ZhiweiYan-96/68/base -> origin/gh/ZhiweiYan-96/68/base 2025-12-04T09:21:03.9771522Z * [new branch] gh/ZhiweiYan-96/68/head -> origin/gh/ZhiweiYan-96/68/head 2025-12-04T09:21:03.9772184Z * [new branch] gh/ZhiweiYan-96/68/orig -> origin/gh/ZhiweiYan-96/68/orig 2025-12-04T09:21:03.9772905Z * [new branch] gh/aakhundov/1/base -> origin/gh/aakhundov/1/base 2025-12-04T09:21:03.9773560Z * [new branch] gh/aakhundov/1/head -> origin/gh/aakhundov/1/head 2025-12-04T09:21:03.9774223Z * [new branch] gh/aakhundov/2/base -> origin/gh/aakhundov/2/base 2025-12-04T09:21:03.9774958Z * [new branch] gh/aakhundov/2/head -> origin/gh/aakhundov/2/head 2025-12-04T09:21:03.9775990Z * [new branch] gh/aditew01/openblas -> origin/gh/aditew01/openblas 2025-12-04T09:21:03.9776667Z * [new branch] gh/aditew01/sbgemm -> origin/gh/aditew01/sbgemm 2025-12-04T09:21:03.9777321Z * [new branch] gh/aditew01/vecbf16 -> origin/gh/aditew01/vecbf16 2025-12-04T09:21:03.9777948Z * [new branch] gh/albanD/4/base -> origin/gh/albanD/4/base 2025-12-04T09:21:03.9778574Z * [new branch] gh/albanD/4/head -> origin/gh/albanD/4/head 2025-12-04T09:21:03.9779194Z * [new branch] gh/albanD/4/orig -> origin/gh/albanD/4/orig 2025-12-04T09:21:03.9780188Z * [new branch] gh/alexbrauckmann/paddedtensor_faketensor_init -> origin/gh/alexbrauckmann/paddedtensor_faketensor_init 2025-12-04T09:21:03.9781304Z * [new branch] gh/alexsamardzic/12/base -> origin/gh/alexsamardzic/12/base 2025-12-04T09:21:03.9782118Z * [new branch] gh/alexsamardzic/12/head -> origin/gh/alexsamardzic/12/head 2025-12-04T09:21:03.9782843Z * [new branch] gh/alexsamardzic/12/orig -> origin/gh/alexsamardzic/12/orig 2025-12-04T09:21:03.9783566Z * [new branch] gh/alexsamardzic/14/base -> origin/gh/alexsamardzic/14/base 2025-12-04T09:21:03.9784324Z * [new branch] gh/alexsamardzic/14/head -> origin/gh/alexsamardzic/14/head 2025-12-04T09:21:03.9785225Z * [new branch] gh/alexsamardzic/14/orig -> origin/gh/alexsamardzic/14/orig 2025-12-04T09:21:03.9785945Z * [new branch] gh/alexsamardzic/15/base -> origin/gh/alexsamardzic/15/base 2025-12-04T09:21:03.9786840Z * [new branch] gh/alexsamardzic/15/head -> origin/gh/alexsamardzic/15/head 2025-12-04T09:21:03.9787639Z * [new branch] gh/alexsamardzic/15/orig -> origin/gh/alexsamardzic/15/orig 2025-12-04T09:21:03.9788725Z * [new branch] gh/amjames/18/base -> origin/gh/amjames/18/base 2025-12-04T09:21:03.9789925Z * [new branch] gh/amjames/18/head -> origin/gh/amjames/18/head 2025-12-04T09:21:03.9791148Z * [new branch] gh/amjames/18/orig -> origin/gh/amjames/18/orig 2025-12-04T09:21:03.9793464Z * [new branch] gh/andrewor14/35/base -> origin/gh/andrewor14/35/base 2025-12-04T09:21:03.9794723Z * [new branch] gh/andrewor14/35/head -> origin/gh/andrewor14/35/head 2025-12-04T09:21:03.9796054Z * [new branch] gh/andrewor14/35/orig -> origin/gh/andrewor14/35/orig 2025-12-04T09:21:03.9797969Z * [new branch] gh/andrewor14/50/base -> origin/gh/andrewor14/50/base 2025-12-04T09:21:03.9799348Z * [new branch] gh/andrewor14/50/head -> origin/gh/andrewor14/50/head 2025-12-04T09:21:03.9800637Z * [new branch] gh/andrewor14/50/orig -> origin/gh/andrewor14/50/orig 2025-12-04T09:21:03.9803656Z * [new branch] gh/andyanwang/30/base -> origin/gh/andyanwang/30/base 2025-12-04T09:21:03.9805129Z * [new branch] gh/andyanwang/30/orig -> origin/gh/andyanwang/30/orig 2025-12-04T09:21:03.9806845Z * [new branch] gh/andyanwang/31/base -> origin/gh/andyanwang/31/base 2025-12-04T09:21:03.9808331Z * [new branch] gh/andyanwang/31/orig -> origin/gh/andyanwang/31/orig 2025-12-04T09:21:03.9810096Z * [new branch] gh/andyanwang/39/base -> origin/gh/andyanwang/39/base 2025-12-04T09:21:03.9811414Z * [new branch] gh/andyanwang/39/head -> origin/gh/andyanwang/39/head 2025-12-04T09:21:03.9812687Z * [new branch] gh/andyanwang/39/orig -> origin/gh/andyanwang/39/orig 2025-12-04T09:21:03.9814706Z * [new branch] gh/andyanwang/42/base -> origin/gh/andyanwang/42/base 2025-12-04T09:21:03.9815848Z * [new branch] gh/andyanwang/42/head -> origin/gh/andyanwang/42/head 2025-12-04T09:21:03.9817049Z * [new branch] gh/andyanwang/42/orig -> origin/gh/andyanwang/42/orig 2025-12-04T09:21:03.9818830Z * [new branch] gh/andyanwang/45/base -> origin/gh/andyanwang/45/base 2025-12-04T09:21:03.9820252Z * [new branch] gh/andyanwang/45/head -> origin/gh/andyanwang/45/head 2025-12-04T09:21:03.9821394Z * [new branch] gh/andyanwang/45/orig -> origin/gh/andyanwang/45/orig 2025-12-04T09:21:03.9823398Z * [new branch] gh/angelayi/107/base -> origin/gh/angelayi/107/base 2025-12-04T09:21:03.9824546Z * [new branch] gh/angelayi/107/head -> origin/gh/angelayi/107/head 2025-12-04T09:21:03.9826245Z * [new branch] gh/angelayi/114/base -> origin/gh/angelayi/114/base 2025-12-04T09:21:03.9827952Z * [new branch] gh/angelayi/114/head -> origin/gh/angelayi/114/head 2025-12-04T09:21:03.9829134Z * [new branch] gh/angelayi/114/orig -> origin/gh/angelayi/114/orig 2025-12-04T09:21:03.9830822Z * [new branch] gh/angelayi/116/base -> origin/gh/angelayi/116/base 2025-12-04T09:21:03.9831992Z * [new branch] gh/angelayi/116/head -> origin/gh/angelayi/116/head 2025-12-04T09:21:03.9833242Z * [new branch] gh/angelayi/116/orig -> origin/gh/angelayi/116/orig 2025-12-04T09:21:03.9835076Z * [new branch] gh/angelayi/122/base -> origin/gh/angelayi/122/base 2025-12-04T09:21:03.9836309Z * [new branch] gh/angelayi/122/head -> origin/gh/angelayi/122/head 2025-12-04T09:21:03.9837650Z * [new branch] gh/angelayi/122/orig -> origin/gh/angelayi/122/orig 2025-12-04T09:21:03.9842883Z * [new branch] gh/angelayi/124/base -> origin/gh/angelayi/124/base 2025-12-04T09:21:03.9843672Z * [new branch] gh/angelayi/124/head -> origin/gh/angelayi/124/head 2025-12-04T09:21:03.9844427Z * [new branch] gh/angelayi/124/orig -> origin/gh/angelayi/124/orig 2025-12-04T09:21:03.9845099Z * [new branch] gh/angelayi/128/base -> origin/gh/angelayi/128/base 2025-12-04T09:21:03.9845808Z * [new branch] gh/angelayi/128/head -> origin/gh/angelayi/128/head 2025-12-04T09:21:03.9846653Z * [new branch] gh/angelayi/128/orig -> origin/gh/angelayi/128/orig 2025-12-04T09:21:03.9848383Z * [new branch] gh/angelayi/131/base -> origin/gh/angelayi/131/base 2025-12-04T09:21:03.9849737Z * [new branch] gh/angelayi/131/head -> origin/gh/angelayi/131/head 2025-12-04T09:21:03.9850917Z * [new branch] gh/angelayi/131/orig -> origin/gh/angelayi/131/orig 2025-12-04T09:21:03.9852858Z * [new branch] gh/angelayi/132/base -> origin/gh/angelayi/132/base 2025-12-04T09:21:03.9854348Z * [new branch] gh/angelayi/132/head -> origin/gh/angelayi/132/head 2025-12-04T09:21:03.9855732Z * [new branch] gh/angelayi/132/orig -> origin/gh/angelayi/132/orig 2025-12-04T09:21:03.9857295Z * [new branch] gh/angelayi/133/base -> origin/gh/angelayi/133/base 2025-12-04T09:21:03.9858455Z * [new branch] gh/angelayi/133/head -> origin/gh/angelayi/133/head 2025-12-04T09:21:03.9859675Z * [new branch] gh/angelayi/133/orig -> origin/gh/angelayi/133/orig 2025-12-04T09:21:03.9861626Z * [new branch] gh/angelayi/134/base -> origin/gh/angelayi/134/base 2025-12-04T09:21:03.9863029Z * [new branch] gh/angelayi/134/head -> origin/gh/angelayi/134/head 2025-12-04T09:21:03.9864206Z * [new branch] gh/angelayi/134/orig -> origin/gh/angelayi/134/orig 2025-12-04T09:21:03.9866092Z * [new branch] gh/angelayi/135/base -> origin/gh/angelayi/135/base 2025-12-04T09:21:03.9867855Z * [new branch] gh/angelayi/135/head -> origin/gh/angelayi/135/head 2025-12-04T09:21:03.9869353Z * [new branch] gh/angelayi/135/orig -> origin/gh/angelayi/135/orig 2025-12-04T09:21:03.9871010Z * [new branch] gh/angelayi/136/base -> origin/gh/angelayi/136/base 2025-12-04T09:21:03.9872334Z * [new branch] gh/angelayi/136/head -> origin/gh/angelayi/136/head 2025-12-04T09:21:03.9873567Z * [new branch] gh/angelayi/136/orig -> origin/gh/angelayi/136/orig 2025-12-04T09:21:03.9875323Z * [new branch] gh/angelayi/137/base -> origin/gh/angelayi/137/base 2025-12-04T09:21:03.9876424Z * [new branch] gh/angelayi/137/head -> origin/gh/angelayi/137/head 2025-12-04T09:21:03.9878028Z * [new branch] gh/angelayi/137/orig -> origin/gh/angelayi/137/orig 2025-12-04T09:21:03.9879692Z * [new branch] gh/angelayi/138/base -> origin/gh/angelayi/138/base 2025-12-04T09:21:03.9880788Z * [new branch] gh/angelayi/138/head -> origin/gh/angelayi/138/head 2025-12-04T09:21:03.9881967Z * [new branch] gh/angelayi/138/orig -> origin/gh/angelayi/138/orig 2025-12-04T09:21:03.9883595Z * [new branch] gh/angelayi/139/base -> origin/gh/angelayi/139/base 2025-12-04T09:21:03.9884784Z * [new branch] gh/angelayi/139/head -> origin/gh/angelayi/139/head 2025-12-04T09:21:03.9885978Z * [new branch] gh/angelayi/139/orig -> origin/gh/angelayi/139/orig 2025-12-04T09:21:03.9887711Z * [new branch] gh/angelayi/140/base -> origin/gh/angelayi/140/base 2025-12-04T09:21:03.9889088Z * [new branch] gh/angelayi/140/head -> origin/gh/angelayi/140/head 2025-12-04T09:21:03.9890325Z * [new branch] gh/angelayi/140/orig -> origin/gh/angelayi/140/orig 2025-12-04T09:21:03.9892496Z * [new branch] gh/angelayi/141/base -> origin/gh/angelayi/141/base 2025-12-04T09:21:03.9893642Z * [new branch] gh/angelayi/141/head -> origin/gh/angelayi/141/head 2025-12-04T09:21:03.9894853Z * [new branch] gh/angelayi/141/orig -> origin/gh/angelayi/141/orig 2025-12-04T09:21:03.9896532Z * [new branch] gh/angelayi/142/base -> origin/gh/angelayi/142/base 2025-12-04T09:21:03.9897683Z * [new branch] gh/angelayi/142/head -> origin/gh/angelayi/142/head 2025-12-04T09:21:03.9898884Z * [new branch] gh/angelayi/142/orig -> origin/gh/angelayi/142/orig 2025-12-04T09:21:03.9906366Z * [new branch] gh/angelayi/143/base -> origin/gh/angelayi/143/base 2025-12-04T09:21:03.9907760Z * [new branch] gh/angelayi/143/head -> origin/gh/angelayi/143/head 2025-12-04T09:21:03.9909003Z * [new branch] gh/angelayi/143/orig -> origin/gh/angelayi/143/orig 2025-12-04T09:21:03.9910897Z * [new branch] gh/angelayi/144/base -> origin/gh/angelayi/144/base 2025-12-04T09:21:03.9912364Z * [new branch] gh/angelayi/144/head -> origin/gh/angelayi/144/head 2025-12-04T09:21:03.9913545Z * [new branch] gh/angelayi/144/orig -> origin/gh/angelayi/144/orig 2025-12-04T09:21:03.9915768Z * [new branch] gh/anijain2305/753/base -> origin/gh/anijain2305/753/base 2025-12-04T09:21:03.9917004Z * [new branch] gh/anijain2305/753/head -> origin/gh/anijain2305/753/head 2025-12-04T09:21:03.9918345Z * [new branch] gh/anijain2305/753/orig -> origin/gh/anijain2305/753/orig 2025-12-04T09:21:03.9920152Z * [new branch] gh/anijain2305/810/base -> origin/gh/anijain2305/810/base 2025-12-04T09:21:03.9921320Z * [new branch] gh/anijain2305/810/head -> origin/gh/anijain2305/810/head 2025-12-04T09:21:03.9922804Z * [new branch] gh/anijain2305/810/orig -> origin/gh/anijain2305/810/orig 2025-12-04T09:21:03.9924214Z * [new branch] gh/anijain2305/854/base -> origin/gh/anijain2305/854/base 2025-12-04T09:21:03.9925468Z * [new branch] gh/anijain2305/854/head -> origin/gh/anijain2305/854/head 2025-12-04T09:21:03.9926640Z * [new branch] gh/anijain2305/854/orig -> origin/gh/anijain2305/854/orig 2025-12-04T09:21:03.9928500Z * [new branch] gh/anijain2305/864/base -> origin/gh/anijain2305/864/base 2025-12-04T09:21:03.9929664Z * [new branch] gh/anijain2305/864/head -> origin/gh/anijain2305/864/head 2025-12-04T09:21:03.9930874Z * [new branch] gh/anijain2305/864/orig -> origin/gh/anijain2305/864/orig 2025-12-04T09:21:03.9932583Z * [new branch] gh/anijain2305/870/base -> origin/gh/anijain2305/870/base 2025-12-04T09:21:03.9933690Z * [new branch] gh/anijain2305/870/head -> origin/gh/anijain2305/870/head 2025-12-04T09:21:03.9934892Z * [new branch] gh/anijain2305/870/orig -> origin/gh/anijain2305/870/orig 2025-12-04T09:21:03.9936641Z * [new branch] gh/anijain2305/873/base -> origin/gh/anijain2305/873/base 2025-12-04T09:21:03.9937743Z * [new branch] gh/anijain2305/873/head -> origin/gh/anijain2305/873/head 2025-12-04T09:21:03.9938924Z * [new branch] gh/anijain2305/873/orig -> origin/gh/anijain2305/873/orig 2025-12-04T09:21:03.9940648Z * [new branch] gh/anijain2305/894/base -> origin/gh/anijain2305/894/base 2025-12-04T09:21:03.9941800Z * [new branch] gh/anijain2305/894/head -> origin/gh/anijain2305/894/head 2025-12-04T09:21:03.9942991Z * [new branch] gh/anijain2305/894/orig -> origin/gh/anijain2305/894/orig 2025-12-04T09:21:03.9944730Z * [new branch] gh/anijain2305/895/base -> origin/gh/anijain2305/895/base 2025-12-04T09:21:03.9945973Z * [new branch] gh/anijain2305/895/head -> origin/gh/anijain2305/895/head 2025-12-04T09:21:03.9947639Z * [new branch] gh/anijain2305/895/orig -> origin/gh/anijain2305/895/orig 2025-12-04T09:21:03.9949344Z * [new branch] gh/anijain2305/910/base -> origin/gh/anijain2305/910/base 2025-12-04T09:21:03.9950559Z * [new branch] gh/anijain2305/910/head -> origin/gh/anijain2305/910/head 2025-12-04T09:21:03.9951854Z * [new branch] gh/anijain2305/910/orig -> origin/gh/anijain2305/910/orig 2025-12-04T09:21:03.9953648Z * [new branch] gh/anijain2305/919/base -> origin/gh/anijain2305/919/base 2025-12-04T09:21:03.9954992Z * [new branch] gh/anijain2305/919/head -> origin/gh/anijain2305/919/head 2025-12-04T09:21:03.9956241Z * [new branch] gh/anijain2305/919/orig -> origin/gh/anijain2305/919/orig 2025-12-04T09:21:03.9957992Z * [new branch] gh/anijain2305/922/base -> origin/gh/anijain2305/922/base 2025-12-04T09:21:03.9959366Z * [new branch] gh/anijain2305/922/head -> origin/gh/anijain2305/922/head 2025-12-04T09:21:03.9960579Z * [new branch] gh/anijain2305/922/orig -> origin/gh/anijain2305/922/orig 2025-12-04T09:21:03.9962433Z * [new branch] gh/anijain2305/932/base -> origin/gh/anijain2305/932/base 2025-12-04T09:21:03.9968987Z * [new branch] gh/anijain2305/932/head -> origin/gh/anijain2305/932/head 2025-12-04T09:21:03.9970298Z * [new branch] gh/anijain2305/932/orig -> origin/gh/anijain2305/932/orig 2025-12-04T09:21:03.9971992Z * [new branch] gh/anijain2305/940/base -> origin/gh/anijain2305/940/base 2025-12-04T09:21:03.9973230Z * [new branch] gh/anijain2305/940/head -> origin/gh/anijain2305/940/head 2025-12-04T09:21:03.9974463Z * [new branch] gh/anijain2305/940/orig -> origin/gh/anijain2305/940/orig 2025-12-04T09:21:03.9976315Z * [new branch] gh/anijain2305/941/base -> origin/gh/anijain2305/941/base 2025-12-04T09:21:03.9977500Z * [new branch] gh/anijain2305/941/head -> origin/gh/anijain2305/941/head 2025-12-04T09:21:03.9978718Z * [new branch] gh/anijain2305/941/orig -> origin/gh/anijain2305/941/orig 2025-12-04T09:21:03.9980383Z * [new branch] gh/anijain2305/942/base -> origin/gh/anijain2305/942/base 2025-12-04T09:21:03.9981579Z * [new branch] gh/anijain2305/942/head -> origin/gh/anijain2305/942/head 2025-12-04T09:21:03.9982861Z * [new branch] gh/anijain2305/942/orig -> origin/gh/anijain2305/942/orig 2025-12-04T09:21:03.9984905Z * [new branch] gh/anijain2305/943/base -> origin/gh/anijain2305/943/base 2025-12-04T09:21:03.9986082Z * [new branch] gh/anijain2305/943/head -> origin/gh/anijain2305/943/head 2025-12-04T09:21:03.9987650Z * [new branch] gh/anijain2305/943/orig -> origin/gh/anijain2305/943/orig 2025-12-04T09:21:03.9989932Z * [new branch] gh/anijain2305/944/base -> origin/gh/anijain2305/944/base 2025-12-04T09:21:03.9991161Z * [new branch] gh/anijain2305/944/head -> origin/gh/anijain2305/944/head 2025-12-04T09:21:03.9993131Z * [new branch] gh/anijain2305/944/orig -> origin/gh/anijain2305/944/orig 2025-12-04T09:21:03.9994868Z * [new branch] gh/anijain2305/945/base -> origin/gh/anijain2305/945/base 2025-12-04T09:21:03.9996115Z * [new branch] gh/anijain2305/945/head -> origin/gh/anijain2305/945/head 2025-12-04T09:21:03.9997344Z * [new branch] gh/anijain2305/945/orig -> origin/gh/anijain2305/945/orig 2025-12-04T09:21:03.9999305Z * [new branch] gh/anijain2305/946/base -> origin/gh/anijain2305/946/base 2025-12-04T09:21:04.0000443Z * [new branch] gh/anijain2305/946/head -> origin/gh/anijain2305/946/head 2025-12-04T09:21:04.0002596Z * [new branch] gh/anijain2305/946/orig -> origin/gh/anijain2305/946/orig 2025-12-04T09:21:04.0004601Z * [new branch] gh/anijain2305/947/base -> origin/gh/anijain2305/947/base 2025-12-04T09:21:04.0005539Z * [new branch] gh/anijain2305/947/head -> origin/gh/anijain2305/947/head 2025-12-04T09:21:04.0006770Z * [new branch] gh/anijain2305/947/orig -> origin/gh/anijain2305/947/orig 2025-12-04T09:21:04.0008504Z * [new branch] gh/anijain2305/948/base -> origin/gh/anijain2305/948/base 2025-12-04T09:21:04.0009684Z * [new branch] gh/anijain2305/948/head -> origin/gh/anijain2305/948/head 2025-12-04T09:21:04.0010921Z * [new branch] gh/anijain2305/948/orig -> origin/gh/anijain2305/948/orig 2025-12-04T09:21:04.0012681Z * [new branch] gh/anijain2305/949/base -> origin/gh/anijain2305/949/base 2025-12-04T09:21:04.0013953Z * [new branch] gh/anijain2305/949/head -> origin/gh/anijain2305/949/head 2025-12-04T09:21:04.0015165Z * [new branch] gh/anijain2305/949/orig -> origin/gh/anijain2305/949/orig 2025-12-04T09:21:04.0016859Z * [new branch] gh/anijain2305/950/base -> origin/gh/anijain2305/950/base 2025-12-04T09:21:04.0018067Z * [new branch] gh/anijain2305/950/head -> origin/gh/anijain2305/950/head 2025-12-04T09:21:04.0019274Z * [new branch] gh/anijain2305/950/orig -> origin/gh/anijain2305/950/orig 2025-12-04T09:21:04.0021093Z * [new branch] gh/anijain2305/951/base -> origin/gh/anijain2305/951/base 2025-12-04T09:21:04.0022278Z * [new branch] gh/anijain2305/951/head -> origin/gh/anijain2305/951/head 2025-12-04T09:21:04.0023490Z * [new branch] gh/anijain2305/951/orig -> origin/gh/anijain2305/951/orig 2025-12-04T09:21:04.0025202Z * [new branch] gh/anijain2305/952/base -> origin/gh/anijain2305/952/base 2025-12-04T09:21:04.0031047Z * [new branch] gh/anijain2305/952/head -> origin/gh/anijain2305/952/head 2025-12-04T09:21:04.0031801Z * [new branch] gh/anijain2305/952/orig -> origin/gh/anijain2305/952/orig 2025-12-04T09:21:04.0032485Z * [new branch] gh/anijain2305/953/base -> origin/gh/anijain2305/953/base 2025-12-04T09:21:04.0033175Z * [new branch] gh/anijain2305/953/head -> origin/gh/anijain2305/953/head 2025-12-04T09:21:04.0033872Z * [new branch] gh/anijain2305/953/orig -> origin/gh/anijain2305/953/orig 2025-12-04T09:21:04.0034588Z * [new branch] gh/anijain2305/954/base -> origin/gh/anijain2305/954/base 2025-12-04T09:21:04.0035459Z * [new branch] gh/anijain2305/954/head -> origin/gh/anijain2305/954/head 2025-12-04T09:21:04.0036625Z * [new branch] gh/anijain2305/954/orig -> origin/gh/anijain2305/954/orig 2025-12-04T09:21:04.0038826Z * [new branch] gh/anijain2305/955/base -> origin/gh/anijain2305/955/base 2025-12-04T09:21:04.0039857Z * [new branch] gh/anijain2305/955/head -> origin/gh/anijain2305/955/head 2025-12-04T09:21:04.0041084Z * [new branch] gh/anijain2305/955/orig -> origin/gh/anijain2305/955/orig 2025-12-04T09:21:04.0042964Z * [new branch] gh/anijain2305/956/base -> origin/gh/anijain2305/956/base 2025-12-04T09:21:04.0044151Z * [new branch] gh/anijain2305/956/head -> origin/gh/anijain2305/956/head 2025-12-04T09:21:04.0045369Z * [new branch] gh/anijain2305/956/orig -> origin/gh/anijain2305/956/orig 2025-12-04T09:21:04.0047167Z * [new branch] gh/anijain2305/957/base -> origin/gh/anijain2305/957/base 2025-12-04T09:21:04.0048350Z * [new branch] gh/anijain2305/957/head -> origin/gh/anijain2305/957/head 2025-12-04T09:21:04.0049524Z * [new branch] gh/anijain2305/957/orig -> origin/gh/anijain2305/957/orig 2025-12-04T09:21:04.0051251Z * [new branch] gh/anijain2305/958/base -> origin/gh/anijain2305/958/base 2025-12-04T09:21:04.0052583Z * [new branch] gh/anijain2305/958/head -> origin/gh/anijain2305/958/head 2025-12-04T09:21:04.0053708Z * [new branch] gh/anijain2305/958/orig -> origin/gh/anijain2305/958/orig 2025-12-04T09:21:04.0055553Z * [new branch] gh/anijain2305/959/base -> origin/gh/anijain2305/959/base 2025-12-04T09:21:04.0056700Z * [new branch] gh/anijain2305/959/head -> origin/gh/anijain2305/959/head 2025-12-04T09:21:04.0057919Z * [new branch] gh/anijain2305/959/orig -> origin/gh/anijain2305/959/orig 2025-12-04T09:21:04.0059651Z * [new branch] gh/anijain2305/960/base -> origin/gh/anijain2305/960/base 2025-12-04T09:21:04.0060850Z * [new branch] gh/anijain2305/960/head -> origin/gh/anijain2305/960/head 2025-12-04T09:21:04.0062063Z * [new branch] gh/anijain2305/960/orig -> origin/gh/anijain2305/960/orig 2025-12-04T09:21:04.0063831Z * [new branch] gh/anijain2305/961/base -> origin/gh/anijain2305/961/base 2025-12-04T09:21:04.0064980Z * [new branch] gh/anijain2305/961/head -> origin/gh/anijain2305/961/head 2025-12-04T09:21:04.0066143Z * [new branch] gh/anijain2305/961/orig -> origin/gh/anijain2305/961/orig 2025-12-04T09:21:04.0068324Z * [new branch] gh/anijain2305/962/base -> origin/gh/anijain2305/962/base 2025-12-04T09:21:04.0069543Z * [new branch] gh/anijain2305/962/head -> origin/gh/anijain2305/962/head 2025-12-04T09:21:04.0070825Z * [new branch] gh/anijain2305/962/orig -> origin/gh/anijain2305/962/orig 2025-12-04T09:21:04.0073060Z * [new branch] gh/anijain2305/963/base -> origin/gh/anijain2305/963/base 2025-12-04T09:21:04.0074515Z * [new branch] gh/anijain2305/963/head -> origin/gh/anijain2305/963/head 2025-12-04T09:21:04.0075812Z * [new branch] gh/anijain2305/963/orig -> origin/gh/anijain2305/963/orig 2025-12-04T09:21:04.0077839Z * [new branch] gh/anijain2305/964/base -> origin/gh/anijain2305/964/base 2025-12-04T09:21:04.0078942Z * [new branch] gh/anijain2305/964/head -> origin/gh/anijain2305/964/head 2025-12-04T09:21:04.0080106Z * [new branch] gh/anijain2305/964/orig -> origin/gh/anijain2305/964/orig 2025-12-04T09:21:04.0081882Z * [new branch] gh/anijain2305/965/base -> origin/gh/anijain2305/965/base 2025-12-04T09:21:04.0083097Z * [new branch] gh/anijain2305/965/head -> origin/gh/anijain2305/965/head 2025-12-04T09:21:04.0084404Z * [new branch] gh/anijain2305/965/orig -> origin/gh/anijain2305/965/orig 2025-12-04T09:21:04.0085941Z * [new branch] gh/anijain2305/966/base -> origin/gh/anijain2305/966/base 2025-12-04T09:21:04.0087110Z * [new branch] gh/anijain2305/966/head -> origin/gh/anijain2305/966/head 2025-12-04T09:21:04.0093103Z * [new branch] gh/anijain2305/966/orig -> origin/gh/anijain2305/966/orig 2025-12-04T09:21:04.0093859Z * [new branch] gh/anijain2305/967/base -> origin/gh/anijain2305/967/base 2025-12-04T09:21:04.0094539Z * [new branch] gh/anijain2305/967/head -> origin/gh/anijain2305/967/head 2025-12-04T09:21:04.0095319Z * [new branch] gh/anijain2305/967/orig -> origin/gh/anijain2305/967/orig 2025-12-04T09:21:04.0096010Z * [new branch] gh/anijain2305/968/base -> origin/gh/anijain2305/968/base 2025-12-04T09:21:04.0096785Z * [new branch] gh/anijain2305/968/head -> origin/gh/anijain2305/968/head 2025-12-04T09:21:04.0098008Z * [new branch] gh/anijain2305/968/orig -> origin/gh/anijain2305/968/orig 2025-12-04T09:21:04.0099663Z * [new branch] gh/anijain2305/969/base -> origin/gh/anijain2305/969/base 2025-12-04T09:21:04.0101318Z * [new branch] gh/anijain2305/969/head -> origin/gh/anijain2305/969/head 2025-12-04T09:21:04.0103449Z * [new branch] gh/anijain2305/969/orig -> origin/gh/anijain2305/969/orig 2025-12-04T09:21:04.0104999Z * [new branch] gh/anijain2305/970/base -> origin/gh/anijain2305/970/base 2025-12-04T09:21:04.0106290Z * [new branch] gh/anijain2305/970/head -> origin/gh/anijain2305/970/head 2025-12-04T09:21:04.0107741Z * [new branch] gh/anijain2305/970/orig -> origin/gh/anijain2305/970/orig 2025-12-04T09:21:04.0110024Z * [new branch] gh/anjali411/216/base -> origin/gh/anjali411/216/base 2025-12-04T09:21:04.0111222Z * [new branch] gh/anjali411/216/head -> origin/gh/anjali411/216/head 2025-12-04T09:21:04.0112452Z * [new branch] gh/anjali411/216/orig -> origin/gh/anjali411/216/orig 2025-12-04T09:21:04.0114667Z * [new branch] gh/anshul-si/1/base -> origin/gh/anshul-si/1/base 2025-12-04T09:21:04.0115876Z * [new branch] gh/anshul-si/1/head -> origin/gh/anshul-si/1/head 2025-12-04T09:21:04.0117430Z * [new branch] gh/anshul-si/2/base -> origin/gh/anshul-si/2/base 2025-12-04T09:21:04.0118677Z * [new branch] gh/anshul-si/2/head -> origin/gh/anshul-si/2/head 2025-12-04T09:21:04.0120123Z * [new branch] gh/anshul-si/3/base -> origin/gh/anshul-si/3/base 2025-12-04T09:21:04.0121303Z * [new branch] gh/anshul-si/3/head -> origin/gh/anshul-si/3/head 2025-12-04T09:21:04.0122722Z * [new branch] gh/anshul-si/4/base -> origin/gh/anshul-si/4/base 2025-12-04T09:21:04.0123891Z * [new branch] gh/anshul-si/4/head -> origin/gh/anshul-si/4/head 2025-12-04T09:21:04.0125352Z * [new branch] gh/anshul-si/5/base -> origin/gh/anshul-si/5/base 2025-12-04T09:21:04.0126507Z * [new branch] gh/anshul-si/5/head -> origin/gh/anshul-si/5/head 2025-12-04T09:21:04.0128311Z * [new branch] gh/anshul-si/53/base -> origin/gh/anshul-si/53/base 2025-12-04T09:21:04.0129540Z * [new branch] gh/anshul-si/53/head -> origin/gh/anshul-si/53/head 2025-12-04T09:21:04.0131258Z * [new branch] gh/anshul-si/58/base -> origin/gh/anshul-si/58/base 2025-12-04T09:21:04.0132363Z * [new branch] gh/anshul-si/58/head -> origin/gh/anshul-si/58/head 2025-12-04T09:21:04.0133904Z * [new branch] gh/anshul-si/66/base -> origin/gh/anshul-si/66/base 2025-12-04T09:21:04.0135119Z * [new branch] gh/anshul-si/66/head -> origin/gh/anshul-si/66/head 2025-12-04T09:21:04.0136304Z * [new branch] gh/anshul-si/66/orig -> origin/gh/anshul-si/66/orig 2025-12-04T09:21:04.0137776Z * [new branch] gh/anshul-si/67/base -> origin/gh/anshul-si/67/base 2025-12-04T09:21:04.0139045Z * [new branch] gh/anshul-si/67/head -> origin/gh/anshul-si/67/head 2025-12-04T09:21:04.0140284Z * [new branch] gh/anshul-si/67/orig -> origin/gh/anshul-si/67/orig 2025-12-04T09:21:04.0142079Z * [new branch] gh/anshul-si/68/base -> origin/gh/anshul-si/68/base 2025-12-04T09:21:04.0143169Z * [new branch] gh/anshul-si/68/head -> origin/gh/anshul-si/68/head 2025-12-04T09:21:04.0144335Z * [new branch] gh/anshul-si/68/orig -> origin/gh/anshul-si/68/orig 2025-12-04T09:21:04.0146269Z * [new branch] gh/anshul-si/69/base -> origin/gh/anshul-si/69/base 2025-12-04T09:21:04.0147714Z * [new branch] gh/anshul-si/69/head -> origin/gh/anshul-si/69/head 2025-12-04T09:21:04.0148937Z * [new branch] gh/anshul-si/69/orig -> origin/gh/anshul-si/69/orig 2025-12-04T09:21:04.0156334Z * [new branch] gh/anshul-si/70/base -> origin/gh/anshul-si/70/base 2025-12-04T09:21:04.0157258Z * [new branch] gh/anshul-si/70/head -> origin/gh/anshul-si/70/head 2025-12-04T09:21:04.0158630Z * [new branch] gh/anshul-si/70/orig -> origin/gh/anshul-si/70/orig 2025-12-04T09:21:04.0160271Z * [new branch] gh/anshul-si/71/base -> origin/gh/anshul-si/71/base 2025-12-04T09:21:04.0161629Z * [new branch] gh/anshul-si/71/head -> origin/gh/anshul-si/71/head 2025-12-04T09:21:04.0162840Z * [new branch] gh/anshul-si/71/orig -> origin/gh/anshul-si/71/orig 2025-12-04T09:21:04.0164577Z * [new branch] gh/anshul-si/72/base -> origin/gh/anshul-si/72/base 2025-12-04T09:21:04.0165940Z * [new branch] gh/anshul-si/72/head -> origin/gh/anshul-si/72/head 2025-12-04T09:21:04.0167142Z * [new branch] gh/anshul-si/72/orig -> origin/gh/anshul-si/72/orig 2025-12-04T09:21:04.0168814Z * [new branch] gh/anshul-si/73/base -> origin/gh/anshul-si/73/base 2025-12-04T09:21:04.0170013Z * [new branch] gh/anshul-si/73/head -> origin/gh/anshul-si/73/head 2025-12-04T09:21:04.0171262Z * [new branch] gh/anshul-si/73/orig -> origin/gh/anshul-si/73/orig 2025-12-04T09:21:04.0173396Z * [new branch] gh/aorenste/132/base -> origin/gh/aorenste/132/base 2025-12-04T09:21:04.0174572Z * [new branch] gh/aorenste/132/head -> origin/gh/aorenste/132/head 2025-12-04T09:21:04.0176389Z * [new branch] gh/aorenste/134/base -> origin/gh/aorenste/134/base 2025-12-04T09:21:04.0177856Z * [new branch] gh/aorenste/134/head -> origin/gh/aorenste/134/head 2025-12-04T09:21:04.0179101Z * [new branch] gh/aorenste/134/orig -> origin/gh/aorenste/134/orig 2025-12-04T09:21:04.0180850Z * [new branch] gh/aorenste/139/base -> origin/gh/aorenste/139/base 2025-12-04T09:21:04.0182022Z * [new branch] gh/aorenste/139/head -> origin/gh/aorenste/139/head 2025-12-04T09:21:04.0183206Z * [new branch] gh/aorenste/139/orig -> origin/gh/aorenste/139/orig 2025-12-04T09:21:04.0184846Z * [new branch] gh/aorenste/141/base -> origin/gh/aorenste/141/base 2025-12-04T09:21:04.0185971Z * [new branch] gh/aorenste/141/head -> origin/gh/aorenste/141/head 2025-12-04T09:21:04.0188353Z * [new branch] gh/aorenste/145/base -> origin/gh/aorenste/145/base 2025-12-04T09:21:04.0189581Z * [new branch] gh/aorenste/145/head -> origin/gh/aorenste/145/head 2025-12-04T09:21:04.0191054Z * [new branch] gh/aorenste/145/orig -> origin/gh/aorenste/145/orig 2025-12-04T09:21:04.0192835Z * [new branch] gh/aorenste/146/base -> origin/gh/aorenste/146/base 2025-12-04T09:21:04.0194157Z * [new branch] gh/aorenste/146/head -> origin/gh/aorenste/146/head 2025-12-04T09:21:04.0195585Z * [new branch] gh/aorenste/146/orig -> origin/gh/aorenste/146/orig 2025-12-04T09:21:04.0197373Z * [new branch] gh/aorenste/147/base -> origin/gh/aorenste/147/base 2025-12-04T09:21:04.0198718Z * [new branch] gh/aorenste/147/head -> origin/gh/aorenste/147/head 2025-12-04T09:21:04.0200045Z * [new branch] gh/aorenste/147/orig -> origin/gh/aorenste/147/orig 2025-12-04T09:21:04.0202642Z * [new branch] gh/aorenste/148/base -> origin/gh/aorenste/148/base 2025-12-04T09:21:04.0203898Z * [new branch] gh/aorenste/148/head -> origin/gh/aorenste/148/head 2025-12-04T09:21:04.0205231Z * [new branch] gh/aorenste/148/orig -> origin/gh/aorenste/148/orig 2025-12-04T09:21:04.0207265Z * [new branch] gh/aorenste/149/base -> origin/gh/aorenste/149/base 2025-12-04T09:21:04.0208468Z * [new branch] gh/aorenste/149/head -> origin/gh/aorenste/149/head 2025-12-04T09:21:04.0209679Z * [new branch] gh/aorenste/149/orig -> origin/gh/aorenste/149/orig 2025-12-04T09:21:04.0211435Z * [new branch] gh/aorenste/150/base -> origin/gh/aorenste/150/base 2025-12-04T09:21:04.0212627Z * [new branch] gh/aorenste/150/head -> origin/gh/aorenste/150/head 2025-12-04T09:21:04.0219148Z * [new branch] gh/aorenste/150/orig -> origin/gh/aorenste/150/orig 2025-12-04T09:21:04.0220722Z * [new branch] gh/aorenste/151/base -> origin/gh/aorenste/151/base 2025-12-04T09:21:04.0221947Z * [new branch] gh/aorenste/151/head -> origin/gh/aorenste/151/head 2025-12-04T09:21:04.0223196Z * [new branch] gh/aorenste/151/orig -> origin/gh/aorenste/151/orig 2025-12-04T09:21:04.0225034Z * [new branch] gh/aorenste/152/base -> origin/gh/aorenste/152/base 2025-12-04T09:21:04.0226159Z * [new branch] gh/aorenste/152/head -> origin/gh/aorenste/152/head 2025-12-04T09:21:04.0227717Z * [new branch] gh/aorenste/152/orig -> origin/gh/aorenste/152/orig 2025-12-04T09:21:04.0229329Z * [new branch] gh/aorenste/153/base -> origin/gh/aorenste/153/base 2025-12-04T09:21:04.0230536Z * [new branch] gh/aorenste/153/head -> origin/gh/aorenste/153/head 2025-12-04T09:21:04.0231783Z * [new branch] gh/aorenste/153/orig -> origin/gh/aorenste/153/orig 2025-12-04T09:21:04.0233334Z * [new branch] gh/aorenste/154/base -> origin/gh/aorenste/154/base 2025-12-04T09:21:04.0234566Z * [new branch] gh/aorenste/154/head -> origin/gh/aorenste/154/head 2025-12-04T09:21:04.0235822Z * [new branch] gh/aorenste/154/orig -> origin/gh/aorenste/154/orig 2025-12-04T09:21:04.0237304Z * [new branch] gh/aorenste/155/base -> origin/gh/aorenste/155/base 2025-12-04T09:21:04.0238616Z * [new branch] gh/aorenste/155/head -> origin/gh/aorenste/155/head 2025-12-04T09:21:04.0239750Z * [new branch] gh/aorenste/155/orig -> origin/gh/aorenste/155/orig 2025-12-04T09:21:04.0241218Z * [new branch] gh/aorenste/156/base -> origin/gh/aorenste/156/base 2025-12-04T09:21:04.0242396Z * [new branch] gh/aorenste/156/head -> origin/gh/aorenste/156/head 2025-12-04T09:21:04.0243501Z * [new branch] gh/aorenste/156/orig -> origin/gh/aorenste/156/orig 2025-12-04T09:21:04.0245394Z * [new branch] gh/aorenste/157/base -> origin/gh/aorenste/157/base 2025-12-04T09:21:04.0246641Z * [new branch] gh/aorenste/157/head -> origin/gh/aorenste/157/head 2025-12-04T09:21:04.0247802Z * [new branch] gh/aorenste/157/orig -> origin/gh/aorenste/157/orig 2025-12-04T09:21:04.0249344Z * [new branch] gh/aorenste/158/base -> origin/gh/aorenste/158/base 2025-12-04T09:21:04.0250584Z * [new branch] gh/aorenste/158/head -> origin/gh/aorenste/158/head 2025-12-04T09:21:04.0251772Z * [new branch] gh/aorenste/158/orig -> origin/gh/aorenste/158/orig 2025-12-04T09:21:04.0253278Z * [new branch] gh/aorenste/159/base -> origin/gh/aorenste/159/base 2025-12-04T09:21:04.0254445Z * [new branch] gh/aorenste/159/head -> origin/gh/aorenste/159/head 2025-12-04T09:21:04.0255537Z * [new branch] gh/aorenste/159/orig -> origin/gh/aorenste/159/orig 2025-12-04T09:21:04.0257535Z * [new branch] gh/avikchaudhuri/1/base -> origin/gh/avikchaudhuri/1/base 2025-12-04T09:21:04.0258771Z * [new branch] gh/avikchaudhuri/1/head -> origin/gh/avikchaudhuri/1/head 2025-12-04T09:21:04.0260351Z * [new branch] gh/avikchaudhuri/2/base -> origin/gh/avikchaudhuri/2/base 2025-12-04T09:21:04.0261414Z * [new branch] gh/avikchaudhuri/2/head -> origin/gh/avikchaudhuri/2/head 2025-12-04T09:21:04.0262651Z * [new branch] gh/avikchaudhuri/2/orig -> origin/gh/avikchaudhuri/2/orig 2025-12-04T09:21:04.0265020Z * [new branch] gh/bdhirsh/666/base -> origin/gh/bdhirsh/666/base 2025-12-04T09:21:04.0266280Z * [new branch] gh/bdhirsh/666/head -> origin/gh/bdhirsh/666/head 2025-12-04T09:21:04.0268261Z * [new branch] gh/bdhirsh/666/orig -> origin/gh/bdhirsh/666/orig 2025-12-04T09:21:04.0269540Z * [new branch] gh/bdhirsh/668/base -> origin/gh/bdhirsh/668/base 2025-12-04T09:21:04.0270777Z * [new branch] gh/bdhirsh/668/head -> origin/gh/bdhirsh/668/head 2025-12-04T09:21:04.0271982Z * [new branch] gh/bdhirsh/668/orig -> origin/gh/bdhirsh/668/orig 2025-12-04T09:21:04.0273959Z * [new branch] gh/bdhirsh/669/base -> origin/gh/bdhirsh/669/base 2025-12-04T09:21:04.0280471Z * [new branch] gh/bdhirsh/669/head -> origin/gh/bdhirsh/669/head 2025-12-04T09:21:04.0281163Z * [new branch] gh/bdhirsh/669/orig -> origin/gh/bdhirsh/669/orig 2025-12-04T09:21:04.0281803Z * [new branch] gh/bdhirsh/670/base -> origin/gh/bdhirsh/670/base 2025-12-04T09:21:04.0282443Z * [new branch] gh/bdhirsh/670/head -> origin/gh/bdhirsh/670/head 2025-12-04T09:21:04.0283185Z * [new branch] gh/bdhirsh/670/orig -> origin/gh/bdhirsh/670/orig 2025-12-04T09:21:04.0283853Z * [new branch] gh/bdhirsh/672/base -> origin/gh/bdhirsh/672/base 2025-12-04T09:21:04.0284561Z * [new branch] gh/bdhirsh/672/head -> origin/gh/bdhirsh/672/head 2025-12-04T09:21:04.0285394Z * [new branch] gh/bdhirsh/672/orig -> origin/gh/bdhirsh/672/orig 2025-12-04T09:21:04.0287405Z * [new branch] gh/bdhirsh/675/base -> origin/gh/bdhirsh/675/base 2025-12-04T09:21:04.0288933Z * [new branch] gh/bdhirsh/675/head -> origin/gh/bdhirsh/675/head 2025-12-04T09:21:04.0290080Z * [new branch] gh/bdhirsh/675/orig -> origin/gh/bdhirsh/675/orig 2025-12-04T09:21:04.0291775Z * [new branch] gh/bdhirsh/676/base -> origin/gh/bdhirsh/676/base 2025-12-04T09:21:04.0293054Z * [new branch] gh/bdhirsh/676/head -> origin/gh/bdhirsh/676/head 2025-12-04T09:21:04.0294224Z * [new branch] gh/bdhirsh/676/orig -> origin/gh/bdhirsh/676/orig 2025-12-04T09:21:04.0295850Z * [new branch] gh/bdhirsh/677/base -> origin/gh/bdhirsh/677/base 2025-12-04T09:21:04.0297478Z * [new branch] gh/bdhirsh/677/head -> origin/gh/bdhirsh/677/head 2025-12-04T09:21:04.0298666Z * [new branch] gh/bdhirsh/677/orig -> origin/gh/bdhirsh/677/orig 2025-12-04T09:21:04.0300438Z * [new branch] gh/bdhirsh/678/base -> origin/gh/bdhirsh/678/base 2025-12-04T09:21:04.0303260Z * [new branch] gh/bdhirsh/678/head -> origin/gh/bdhirsh/678/head 2025-12-04T09:21:04.0304515Z * [new branch] gh/bdhirsh/678/orig -> origin/gh/bdhirsh/678/orig 2025-12-04T09:21:04.0306369Z * [new branch] gh/bdhirsh/679/base -> origin/gh/bdhirsh/679/base 2025-12-04T09:21:04.0307909Z * [new branch] gh/bdhirsh/679/head -> origin/gh/bdhirsh/679/head 2025-12-04T09:21:04.0309156Z * [new branch] gh/bdhirsh/679/orig -> origin/gh/bdhirsh/679/orig 2025-12-04T09:21:04.0310906Z * [new branch] gh/bdhirsh/680/base -> origin/gh/bdhirsh/680/base 2025-12-04T09:21:04.0312235Z * [new branch] gh/bdhirsh/680/head -> origin/gh/bdhirsh/680/head 2025-12-04T09:21:04.0313521Z * [new branch] gh/bdhirsh/680/orig -> origin/gh/bdhirsh/680/orig 2025-12-04T09:21:04.0314989Z * [new branch] gh/bdhirsh/681/base -> origin/gh/bdhirsh/681/base 2025-12-04T09:21:04.0316345Z * [new branch] gh/bdhirsh/681/head -> origin/gh/bdhirsh/681/head 2025-12-04T09:21:04.0317713Z * [new branch] gh/bdhirsh/681/orig -> origin/gh/bdhirsh/681/orig 2025-12-04T09:21:04.0320235Z * [new branch] gh/benjaminglass1/101/base -> origin/gh/benjaminglass1/101/base 2025-12-04T09:21:04.0321484Z * [new branch] gh/benjaminglass1/101/head -> origin/gh/benjaminglass1/101/head 2025-12-04T09:21:04.0322763Z * [new branch] gh/benjaminglass1/101/orig -> origin/gh/benjaminglass1/101/orig 2025-12-04T09:21:04.0324357Z * [new branch] gh/benjaminglass1/102/base -> origin/gh/benjaminglass1/102/base 2025-12-04T09:21:04.0325586Z * [new branch] gh/benjaminglass1/102/head -> origin/gh/benjaminglass1/102/head 2025-12-04T09:21:04.0326861Z * [new branch] gh/benjaminglass1/102/orig -> origin/gh/benjaminglass1/102/orig 2025-12-04T09:21:04.0328395Z * [new branch] gh/benjaminglass1/106/base -> origin/gh/benjaminglass1/106/base 2025-12-04T09:21:04.0329595Z * [new branch] gh/benjaminglass1/106/head -> origin/gh/benjaminglass1/106/head 2025-12-04T09:21:04.0330786Z * [new branch] gh/benjaminglass1/106/orig -> origin/gh/benjaminglass1/106/orig 2025-12-04T09:21:04.0332343Z * [new branch] gh/benjaminglass1/107/base -> origin/gh/benjaminglass1/107/base 2025-12-04T09:21:04.0333537Z * [new branch] gh/benjaminglass1/107/head -> origin/gh/benjaminglass1/107/head 2025-12-04T09:21:04.0334793Z * [new branch] gh/benjaminglass1/107/orig -> origin/gh/benjaminglass1/107/orig 2025-12-04T09:21:04.0336485Z * [new branch] gh/benjaminglass1/108/base -> origin/gh/benjaminglass1/108/base 2025-12-04T09:21:04.0337712Z * [new branch] gh/benjaminglass1/108/head -> origin/gh/benjaminglass1/108/head 2025-12-04T09:21:04.0342950Z * [new branch] gh/benjaminglass1/108/orig -> origin/gh/benjaminglass1/108/orig 2025-12-04T09:21:04.0343280Z * [new branch] gh/benjaminglass1/109/base -> origin/gh/benjaminglass1/109/base 2025-12-04T09:21:04.0343616Z * [new branch] gh/benjaminglass1/109/head -> origin/gh/benjaminglass1/109/head 2025-12-04T09:21:04.0343923Z * [new branch] gh/benjaminglass1/109/orig -> origin/gh/benjaminglass1/109/orig 2025-12-04T09:21:04.0345349Z * [new branch] gh/benjaminglass1/97/base -> origin/gh/benjaminglass1/97/base 2025-12-04T09:21:04.0346713Z * [new branch] gh/benjaminglass1/97/head -> origin/gh/benjaminglass1/97/head 2025-12-04T09:21:04.0348028Z * [new branch] gh/benjaminglass1/97/orig -> origin/gh/benjaminglass1/97/orig 2025-12-04T09:21:04.0349940Z * [new branch] gh/bobrenjc93/570/base -> origin/gh/bobrenjc93/570/base 2025-12-04T09:21:04.0351216Z * [new branch] gh/bobrenjc93/570/head -> origin/gh/bobrenjc93/570/head 2025-12-04T09:21:04.0352498Z * [new branch] gh/bobrenjc93/570/orig -> origin/gh/bobrenjc93/570/orig 2025-12-04T09:21:04.0354142Z * [new branch] gh/bobrenjc93/604/base -> origin/gh/bobrenjc93/604/base 2025-12-04T09:21:04.0355391Z * [new branch] gh/bobrenjc93/604/head -> origin/gh/bobrenjc93/604/head 2025-12-04T09:21:04.0356654Z * [new branch] gh/bobrenjc93/604/orig -> origin/gh/bobrenjc93/604/orig 2025-12-04T09:21:04.0358394Z * [new branch] gh/bobrenjc93/638/base -> origin/gh/bobrenjc93/638/base 2025-12-04T09:21:04.0359580Z * [new branch] gh/bobrenjc93/638/head -> origin/gh/bobrenjc93/638/head 2025-12-04T09:21:04.0360796Z * [new branch] gh/bobrenjc93/638/orig -> origin/gh/bobrenjc93/638/orig 2025-12-04T09:21:04.0362421Z * [new branch] gh/bobrenjc93/653/base -> origin/gh/bobrenjc93/653/base 2025-12-04T09:21:04.0363595Z * [new branch] gh/bobrenjc93/653/head -> origin/gh/bobrenjc93/653/head 2025-12-04T09:21:04.0364824Z * [new branch] gh/bobrenjc93/653/orig -> origin/gh/bobrenjc93/653/orig 2025-12-04T09:21:04.0366642Z * [new branch] gh/bobrenjc93/654/base -> origin/gh/bobrenjc93/654/base 2025-12-04T09:21:04.0367769Z * [new branch] gh/bobrenjc93/654/head -> origin/gh/bobrenjc93/654/head 2025-12-04T09:21:04.0368913Z * [new branch] gh/bobrenjc93/654/orig -> origin/gh/bobrenjc93/654/orig 2025-12-04T09:21:04.0370591Z * [new branch] gh/bobrenjc93/657/base -> origin/gh/bobrenjc93/657/base 2025-12-04T09:21:04.0371768Z * [new branch] gh/bobrenjc93/657/head -> origin/gh/bobrenjc93/657/head 2025-12-04T09:21:04.0372954Z * [new branch] gh/bobrenjc93/657/orig -> origin/gh/bobrenjc93/657/orig 2025-12-04T09:21:04.0374571Z * [new branch] gh/bobrenjc93/672/base -> origin/gh/bobrenjc93/672/base 2025-12-04T09:21:04.0375726Z * [new branch] gh/bobrenjc93/672/head -> origin/gh/bobrenjc93/672/head 2025-12-04T09:21:04.0376947Z * [new branch] gh/bobrenjc93/672/orig -> origin/gh/bobrenjc93/672/orig 2025-12-04T09:21:04.0378574Z * [new branch] gh/bobrenjc93/679/base -> origin/gh/bobrenjc93/679/base 2025-12-04T09:21:04.0380015Z * [new branch] gh/bobrenjc93/679/head -> origin/gh/bobrenjc93/679/head 2025-12-04T09:21:04.0381241Z * [new branch] gh/bobrenjc93/679/orig -> origin/gh/bobrenjc93/679/orig 2025-12-04T09:21:04.0382892Z * [new branch] gh/bobrenjc93/680/base -> origin/gh/bobrenjc93/680/base 2025-12-04T09:21:04.0384055Z * [new branch] gh/bobrenjc93/680/head -> origin/gh/bobrenjc93/680/head 2025-12-04T09:21:04.0385249Z * [new branch] gh/bobrenjc93/680/orig -> origin/gh/bobrenjc93/680/orig 2025-12-04T09:21:04.0386944Z * [new branch] gh/bobrenjc93/681/base -> origin/gh/bobrenjc93/681/base 2025-12-04T09:21:04.0388446Z * [new branch] gh/bobrenjc93/681/head -> origin/gh/bobrenjc93/681/head 2025-12-04T09:21:04.0389693Z * [new branch] gh/bobrenjc93/681/orig -> origin/gh/bobrenjc93/681/orig 2025-12-04T09:21:04.0391192Z * [new branch] gh/bobrenjc93/682/base -> origin/gh/bobrenjc93/682/base 2025-12-04T09:21:04.0392424Z * [new branch] gh/bobrenjc93/682/head -> origin/gh/bobrenjc93/682/head 2025-12-04T09:21:04.0393664Z * [new branch] gh/bobrenjc93/682/orig -> origin/gh/bobrenjc93/682/orig 2025-12-04T09:21:04.0395297Z * [new branch] gh/bobrenjc93/683/base -> origin/gh/bobrenjc93/683/base 2025-12-04T09:21:04.0396506Z * [new branch] gh/bobrenjc93/683/head -> origin/gh/bobrenjc93/683/head 2025-12-04T09:21:04.0397756Z * [new branch] gh/bobrenjc93/683/orig -> origin/gh/bobrenjc93/683/orig 2025-12-04T09:21:04.0399394Z * [new branch] gh/bobrenjc93/684/base -> origin/gh/bobrenjc93/684/base 2025-12-04T09:21:04.0406673Z * [new branch] gh/bobrenjc93/684/head -> origin/gh/bobrenjc93/684/head 2025-12-04T09:21:04.0408063Z * [new branch] gh/bobrenjc93/684/orig -> origin/gh/bobrenjc93/684/orig 2025-12-04T09:21:04.0409666Z * [new branch] gh/bobrenjc93/685/base -> origin/gh/bobrenjc93/685/base 2025-12-04T09:21:04.0411171Z * [new branch] gh/bobrenjc93/685/head -> origin/gh/bobrenjc93/685/head 2025-12-04T09:21:04.0412708Z * [new branch] gh/bobrenjc93/685/orig -> origin/gh/bobrenjc93/685/orig 2025-12-04T09:21:04.0414718Z * [new branch] gh/bobrenjc93/686/base -> origin/gh/bobrenjc93/686/base 2025-12-04T09:21:04.0415897Z * [new branch] gh/bobrenjc93/686/head -> origin/gh/bobrenjc93/686/head 2025-12-04T09:21:04.0417120Z * [new branch] gh/bobrenjc93/686/orig -> origin/gh/bobrenjc93/686/orig 2025-12-04T09:21:04.0418604Z * [new branch] gh/bobrenjc93/687/base -> origin/gh/bobrenjc93/687/base 2025-12-04T09:21:04.0419983Z * [new branch] gh/bobrenjc93/687/head -> origin/gh/bobrenjc93/687/head 2025-12-04T09:21:04.0421218Z * [new branch] gh/bobrenjc93/687/orig -> origin/gh/bobrenjc93/687/orig 2025-12-04T09:21:04.0423340Z * [new branch] gh/bobrenjc93/688/base -> origin/gh/bobrenjc93/688/base 2025-12-04T09:21:04.0424552Z * [new branch] gh/bobrenjc93/688/head -> origin/gh/bobrenjc93/688/head 2025-12-04T09:21:04.0425753Z * [new branch] gh/bobrenjc93/688/orig -> origin/gh/bobrenjc93/688/orig 2025-12-04T09:21:04.0427790Z * [new branch] gh/bobrenjc93/689/base -> origin/gh/bobrenjc93/689/base 2025-12-04T09:21:04.0429403Z * [new branch] gh/bobrenjc93/689/head -> origin/gh/bobrenjc93/689/head 2025-12-04T09:21:04.0430657Z * [new branch] gh/bobrenjc93/689/orig -> origin/gh/bobrenjc93/689/orig 2025-12-04T09:21:04.0432263Z * [new branch] gh/bobrenjc93/690/base -> origin/gh/bobrenjc93/690/base 2025-12-04T09:21:04.0433504Z * [new branch] gh/bobrenjc93/690/head -> origin/gh/bobrenjc93/690/head 2025-12-04T09:21:04.0434710Z * [new branch] gh/bobrenjc93/690/orig -> origin/gh/bobrenjc93/690/orig 2025-12-04T09:21:04.0437239Z * [new branch] gh/bobrenjc93/691/base -> origin/gh/bobrenjc93/691/base 2025-12-04T09:21:04.0438803Z * [new branch] gh/bobrenjc93/691/head -> origin/gh/bobrenjc93/691/head 2025-12-04T09:21:04.0440623Z * [new branch] gh/bobrenjc93/691/orig -> origin/gh/bobrenjc93/691/orig 2025-12-04T09:21:04.0443025Z * [new branch] gh/bobrenjc93/692/base -> origin/gh/bobrenjc93/692/base 2025-12-04T09:21:04.0444267Z * [new branch] gh/bobrenjc93/692/head -> origin/gh/bobrenjc93/692/head 2025-12-04T09:21:04.0445518Z * [new branch] gh/bobrenjc93/692/orig -> origin/gh/bobrenjc93/692/orig 2025-12-04T09:21:04.0447082Z * [new branch] gh/bobrenjc93/693/base -> origin/gh/bobrenjc93/693/base 2025-12-04T09:21:04.0448262Z * [new branch] gh/bobrenjc93/693/head -> origin/gh/bobrenjc93/693/head 2025-12-04T09:21:04.0449539Z * [new branch] gh/bobrenjc93/693/orig -> origin/gh/bobrenjc93/693/orig 2025-12-04T09:21:04.0451206Z * [new branch] gh/bobrenjc93/694/base -> origin/gh/bobrenjc93/694/base 2025-12-04T09:21:04.0452408Z * [new branch] gh/bobrenjc93/694/head -> origin/gh/bobrenjc93/694/head 2025-12-04T09:21:04.0455040Z * [new branch] gh/bobrenjc93/694/orig -> origin/gh/bobrenjc93/694/orig 2025-12-04T09:21:04.0455986Z * [new branch] gh/bobrenjc93/695/base -> origin/gh/bobrenjc93/695/base 2025-12-04T09:21:04.0456832Z * [new branch] gh/bobrenjc93/695/head -> origin/gh/bobrenjc93/695/head 2025-12-04T09:21:04.0457606Z * [new branch] gh/bobrenjc93/695/orig -> origin/gh/bobrenjc93/695/orig 2025-12-04T09:21:04.0459884Z * [new branch] gh/c00w/23/base -> origin/gh/c00w/23/base 2025-12-04T09:21:04.0460706Z * [new branch] gh/c00w/23/head -> origin/gh/c00w/23/head 2025-12-04T09:21:04.0462700Z * [new branch] gh/c00w/53/base -> origin/gh/c00w/53/base 2025-12-04T09:21:04.0468574Z * [new branch] gh/c00w/53/head -> origin/gh/c00w/53/head 2025-12-04T09:21:04.0469793Z * [new branch] gh/c00w/53/orig -> origin/gh/c00w/53/orig 2025-12-04T09:21:04.0471334Z * [new branch] gh/c00w/54/base -> origin/gh/c00w/54/base 2025-12-04T09:21:04.0472642Z * [new branch] gh/c00w/54/head -> origin/gh/c00w/54/head 2025-12-04T09:21:04.0473925Z * [new branch] gh/c00w/54/orig -> origin/gh/c00w/54/orig 2025-12-04T09:21:04.0475609Z * [new branch] gh/c00w/56/base -> origin/gh/c00w/56/base 2025-12-04T09:21:04.0476883Z * [new branch] gh/c00w/56/head -> origin/gh/c00w/56/head 2025-12-04T09:21:04.0478079Z * [new branch] gh/c00w/56/orig -> origin/gh/c00w/56/orig 2025-12-04T09:21:04.0479732Z * [new branch] gh/c00w/57/base -> origin/gh/c00w/57/base 2025-12-04T09:21:04.0480929Z * [new branch] gh/c00w/57/head -> origin/gh/c00w/57/head 2025-12-04T09:21:04.0482177Z * [new branch] gh/c00w/57/orig -> origin/gh/c00w/57/orig 2025-12-04T09:21:04.0483888Z * [new branch] gh/c00w/58/base -> origin/gh/c00w/58/base 2025-12-04T09:21:04.0485097Z * [new branch] gh/c00w/58/head -> origin/gh/c00w/58/head 2025-12-04T09:21:04.0486280Z * [new branch] gh/c00w/58/orig -> origin/gh/c00w/58/orig 2025-12-04T09:21:04.0488219Z * [new branch] gh/clee2000/1/base -> origin/gh/clee2000/1/base 2025-12-04T09:21:04.0489487Z * [new branch] gh/clee2000/1/head -> origin/gh/clee2000/1/head 2025-12-04T09:21:04.0490687Z * [new branch] gh/clee2000/1/orig -> origin/gh/clee2000/1/orig 2025-12-04T09:21:04.0492790Z * [new branch] gh/coconutruben/1/base -> origin/gh/coconutruben/1/base 2025-12-04T09:21:04.0494092Z * [new branch] gh/coconutruben/1/head -> origin/gh/coconutruben/1/head 2025-12-04T09:21:04.0496045Z * [new branch] gh/coconutruben/55/base -> origin/gh/coconutruben/55/base 2025-12-04T09:21:04.0497196Z * [new branch] gh/coconutruben/55/head -> origin/gh/coconutruben/55/head 2025-12-04T09:21:04.0498598Z * [new branch] gh/coconutruben/55/orig -> origin/gh/coconutruben/55/orig 2025-12-04T09:21:04.0500316Z * [new branch] gh/coconutruben/57/base -> origin/gh/coconutruben/57/base 2025-12-04T09:21:04.0502644Z * [new branch] gh/coconutruben/57/head -> origin/gh/coconutruben/57/head 2025-12-04T09:21:04.0503924Z * [new branch] gh/coconutruben/57/orig -> origin/gh/coconutruben/57/orig 2025-12-04T09:21:04.0505585Z * [new branch] gh/coconutruben/70/base -> origin/gh/coconutruben/70/base 2025-12-04T09:21:04.0507067Z * [new branch] gh/coconutruben/70/head -> origin/gh/coconutruben/70/head 2025-12-04T09:21:04.0508425Z * [new branch] gh/coconutruben/70/orig -> origin/gh/coconutruben/70/orig 2025-12-04T09:21:04.0509924Z * [new branch] gh/coconutruben/71/base -> origin/gh/coconutruben/71/base 2025-12-04T09:21:04.0511258Z * [new branch] gh/coconutruben/71/head -> origin/gh/coconutruben/71/head 2025-12-04T09:21:04.0512513Z * [new branch] gh/coconutruben/71/orig -> origin/gh/coconutruben/71/orig 2025-12-04T09:21:04.0514007Z * [new branch] gh/coconutruben/72/base -> origin/gh/coconutruben/72/base 2025-12-04T09:21:04.0515254Z * [new branch] gh/coconutruben/72/head -> origin/gh/coconutruben/72/head 2025-12-04T09:21:04.0516641Z * [new branch] gh/coconutruben/72/orig -> origin/gh/coconutruben/72/orig 2025-12-04T09:21:04.0518259Z * [new branch] gh/coconutruben/73/base -> origin/gh/coconutruben/73/base 2025-12-04T09:21:04.0519515Z * [new branch] gh/coconutruben/73/head -> origin/gh/coconutruben/73/head 2025-12-04T09:21:04.0520738Z * [new branch] gh/coconutruben/73/orig -> origin/gh/coconutruben/73/orig 2025-12-04T09:21:04.0522506Z * [new branch] gh/coconutruben/74/base -> origin/gh/coconutruben/74/base 2025-12-04T09:21:04.0523887Z * [new branch] gh/coconutruben/74/head -> origin/gh/coconutruben/74/head 2025-12-04T09:21:04.0525208Z * [new branch] gh/coconutruben/74/orig -> origin/gh/coconutruben/74/orig 2025-12-04T09:21:04.0530437Z * [new branch] gh/coconutruben/79/base -> origin/gh/coconutruben/79/base 2025-12-04T09:21:04.0530833Z * [new branch] gh/coconutruben/79/head -> origin/gh/coconutruben/79/head 2025-12-04T09:21:04.0531122Z * [new branch] gh/coconutruben/79/orig -> origin/gh/coconutruben/79/orig 2025-12-04T09:21:04.0531687Z * [new branch] gh/coconutruben/80/base -> origin/gh/coconutruben/80/base 2025-12-04T09:21:04.0532993Z * [new branch] gh/coconutruben/80/head -> origin/gh/coconutruben/80/head 2025-12-04T09:21:04.0534385Z * [new branch] gh/coconutruben/80/orig -> origin/gh/coconutruben/80/orig 2025-12-04T09:21:04.0536256Z * [new branch] gh/coconutruben/82/base -> origin/gh/coconutruben/82/base 2025-12-04T09:21:04.0537486Z * [new branch] gh/coconutruben/82/head -> origin/gh/coconutruben/82/head 2025-12-04T09:21:04.0538772Z * [new branch] gh/coconutruben/82/orig -> origin/gh/coconutruben/82/orig 2025-12-04T09:21:04.0540577Z * [new branch] gh/coconutruben/83/base -> origin/gh/coconutruben/83/base 2025-12-04T09:21:04.0541750Z * [new branch] gh/coconutruben/83/head -> origin/gh/coconutruben/83/head 2025-12-04T09:21:04.0543254Z * [new branch] gh/coconutruben/83/orig -> origin/gh/coconutruben/83/orig 2025-12-04T09:21:04.0545001Z * [new branch] gh/coconutruben/84/base -> origin/gh/coconutruben/84/base 2025-12-04T09:21:04.0546296Z * [new branch] gh/coconutruben/84/head -> origin/gh/coconutruben/84/head 2025-12-04T09:21:04.0548030Z * [new branch] gh/coconutruben/84/orig -> origin/gh/coconutruben/84/orig 2025-12-04T09:21:04.0549590Z * [new branch] gh/coconutruben/85/base -> origin/gh/coconutruben/85/base 2025-12-04T09:21:04.0550924Z * [new branch] gh/coconutruben/85/head -> origin/gh/coconutruben/85/head 2025-12-04T09:21:04.0552310Z * [new branch] gh/coconutruben/85/orig -> origin/gh/coconutruben/85/orig 2025-12-04T09:21:04.0554010Z * [new branch] gh/coconutruben/86/base -> origin/gh/coconutruben/86/base 2025-12-04T09:21:04.0555290Z * [new branch] gh/coconutruben/86/head -> origin/gh/coconutruben/86/head 2025-12-04T09:21:04.0556608Z * [new branch] gh/coconutruben/86/orig -> origin/gh/coconutruben/86/orig 2025-12-04T09:21:04.0558659Z * [new branch] gh/colinchan15/1/base -> origin/gh/colinchan15/1/base 2025-12-04T09:21:04.0560206Z * [new branch] gh/colinchan15/1/head -> origin/gh/colinchan15/1/head 2025-12-04T09:21:04.0561473Z * [new branch] gh/colinchan15/2/base -> origin/gh/colinchan15/2/base 2025-12-04T09:21:04.0562631Z * [new branch] gh/colinchan15/2/head -> origin/gh/colinchan15/2/head 2025-12-04T09:21:04.0564065Z * [new branch] gh/colinchan15/3/base -> origin/gh/colinchan15/3/base 2025-12-04T09:21:04.0565174Z * [new branch] gh/colinchan15/3/head -> origin/gh/colinchan15/3/head 2025-12-04T09:21:04.0566746Z * [new branch] gh/colinchan15/6/base -> origin/gh/colinchan15/6/base 2025-12-04T09:21:04.0567992Z * [new branch] gh/colinchan15/6/head -> origin/gh/colinchan15/6/head 2025-12-04T09:21:04.0569901Z * [new branch] gh/d4l3k/1/base -> origin/gh/d4l3k/1/base 2025-12-04T09:21:04.0571133Z * [new branch] gh/d4l3k/1/head -> origin/gh/d4l3k/1/head 2025-12-04T09:21:04.0572752Z * [new branch] gh/d4l3k/2/base -> origin/gh/d4l3k/2/base 2025-12-04T09:21:04.0573971Z * [new branch] gh/d4l3k/2/head -> origin/gh/d4l3k/2/head 2025-12-04T09:21:04.0575189Z * [new branch] gh/d4l3k/2/orig -> origin/gh/d4l3k/2/orig 2025-12-04T09:21:04.0576736Z * [new branch] gh/d4l3k/3/base -> origin/gh/d4l3k/3/base 2025-12-04T09:21:04.0577919Z * [new branch] gh/d4l3k/3/head -> origin/gh/d4l3k/3/head 2025-12-04T09:21:04.0579188Z * [new branch] gh/d4l3k/3/orig -> origin/gh/d4l3k/3/orig 2025-12-04T09:21:04.0580737Z * [new branch] gh/d4l3k/4/base -> origin/gh/d4l3k/4/base 2025-12-04T09:21:04.0582046Z * [new branch] gh/d4l3k/4/head -> origin/gh/d4l3k/4/head 2025-12-04T09:21:04.0583218Z * [new branch] gh/d4l3k/4/orig -> origin/gh/d4l3k/4/orig 2025-12-04T09:21:04.0584812Z * [new branch] gh/d4l3k/5/base -> origin/gh/d4l3k/5/base 2025-12-04T09:21:04.0585980Z * [new branch] gh/d4l3k/5/orig -> origin/gh/d4l3k/5/orig 2025-12-04T09:21:04.0593571Z * [new branch] gh/davidberard98/392/base -> origin/gh/davidberard98/392/base 2025-12-04T09:21:04.0593885Z * [new branch] gh/davidberard98/392/head -> origin/gh/davidberard98/392/head 2025-12-04T09:21:04.0594272Z * [new branch] gh/davidberard98/392/orig -> origin/gh/davidberard98/392/orig 2025-12-04T09:21:04.0594595Z * [new branch] gh/davidberard98/399/base -> origin/gh/davidberard98/399/base 2025-12-04T09:21:04.0594891Z * [new branch] gh/davidberard98/399/head -> origin/gh/davidberard98/399/head 2025-12-04T09:21:04.0595724Z * [new branch] gh/davidberard98/399/orig -> origin/gh/davidberard98/399/orig 2025-12-04T09:21:04.0597820Z * [new branch] gh/desertfire/605/base -> origin/gh/desertfire/605/base 2025-12-04T09:21:04.0599047Z * [new branch] gh/desertfire/605/head -> origin/gh/desertfire/605/head 2025-12-04T09:21:04.0600315Z * [new branch] gh/desertfire/605/orig -> origin/gh/desertfire/605/orig 2025-12-04T09:21:04.0602689Z * [new branch] gh/desertfire/606/base -> origin/gh/desertfire/606/base 2025-12-04T09:21:04.0603926Z * [new branch] gh/desertfire/606/head -> origin/gh/desertfire/606/head 2025-12-04T09:21:04.0605346Z * [new branch] gh/desertfire/606/orig -> origin/gh/desertfire/606/orig 2025-12-04T09:21:04.0606979Z * [new branch] gh/desertfire/607/base -> origin/gh/desertfire/607/base 2025-12-04T09:21:04.0608219Z * [new branch] gh/desertfire/607/head -> origin/gh/desertfire/607/head 2025-12-04T09:21:04.0609477Z * [new branch] gh/desertfire/607/orig -> origin/gh/desertfire/607/orig 2025-12-04T09:21:04.0611105Z * [new branch] gh/desertfire/608/base -> origin/gh/desertfire/608/base 2025-12-04T09:21:04.0612306Z * [new branch] gh/desertfire/608/head -> origin/gh/desertfire/608/head 2025-12-04T09:21:04.0613647Z * [new branch] gh/desertfire/608/orig -> origin/gh/desertfire/608/orig 2025-12-04T09:21:04.0615335Z * [new branch] gh/desertfire/609/base -> origin/gh/desertfire/609/base 2025-12-04T09:21:04.0616581Z * [new branch] gh/desertfire/609/head -> origin/gh/desertfire/609/head 2025-12-04T09:21:04.0617766Z * [new branch] gh/desertfire/609/orig -> origin/gh/desertfire/609/orig 2025-12-04T09:21:04.0619551Z * [new branch] gh/desertfire/610/base -> origin/gh/desertfire/610/base 2025-12-04T09:21:04.0620791Z * [new branch] gh/desertfire/610/head -> origin/gh/desertfire/610/head 2025-12-04T09:21:04.0622011Z * [new branch] gh/desertfire/610/orig -> origin/gh/desertfire/610/orig 2025-12-04T09:21:04.0623566Z * [new branch] gh/desertfire/611/base -> origin/gh/desertfire/611/base 2025-12-04T09:21:04.0624815Z * [new branch] gh/desertfire/611/head -> origin/gh/desertfire/611/head 2025-12-04T09:21:04.0626019Z * [new branch] gh/desertfire/611/orig -> origin/gh/desertfire/611/orig 2025-12-04T09:21:04.0628140Z * [new branch] gh/desertfire/612/base -> origin/gh/desertfire/612/base 2025-12-04T09:21:04.0629375Z * [new branch] gh/desertfire/612/head -> origin/gh/desertfire/612/head 2025-12-04T09:21:04.0630531Z * [new branch] gh/desertfire/612/orig -> origin/gh/desertfire/612/orig 2025-12-04T09:21:04.0632319Z * [new branch] gh/desertfire/613/base -> origin/gh/desertfire/613/base 2025-12-04T09:21:04.0633651Z * [new branch] gh/desertfire/613/head -> origin/gh/desertfire/613/head 2025-12-04T09:21:04.0634892Z * [new branch] gh/desertfire/613/orig -> origin/gh/desertfire/613/orig 2025-12-04T09:21:04.0636710Z * [new branch] gh/desertfire/614/base -> origin/gh/desertfire/614/base 2025-12-04T09:21:04.0638040Z * [new branch] gh/desertfire/614/head -> origin/gh/desertfire/614/head 2025-12-04T09:21:04.0639419Z * [new branch] gh/desertfire/614/orig -> origin/gh/desertfire/614/orig 2025-12-04T09:21:04.0641042Z * [new branch] gh/desertfire/615/base -> origin/gh/desertfire/615/base 2025-12-04T09:21:04.0642512Z * [new branch] gh/desertfire/615/head -> origin/gh/desertfire/615/head 2025-12-04T09:21:04.0643693Z * [new branch] gh/desertfire/615/orig -> origin/gh/desertfire/615/orig 2025-12-04T09:21:04.0645140Z * [new branch] gh/desertfire/616/base -> origin/gh/desertfire/616/base 2025-12-04T09:21:04.0646493Z * [new branch] gh/desertfire/616/head -> origin/gh/desertfire/616/head 2025-12-04T09:21:04.0647622Z * [new branch] gh/desertfire/616/orig -> origin/gh/desertfire/616/orig 2025-12-04T09:21:04.0649410Z * [new branch] gh/desertfire/617/base -> origin/gh/desertfire/617/base 2025-12-04T09:21:04.0656358Z * [new branch] gh/desertfire/617/head -> origin/gh/desertfire/617/head 2025-12-04T09:21:04.0657514Z * [new branch] gh/desertfire/617/orig -> origin/gh/desertfire/617/orig 2025-12-04T09:21:04.0659568Z * [new branch] gh/dharakk/1/base -> origin/gh/dharakk/1/base 2025-12-04T09:21:04.0660912Z * [new branch] gh/dharakk/1/head -> origin/gh/dharakk/1/head 2025-12-04T09:21:04.0663040Z * [new branch] gh/drisspg/170/base -> origin/gh/drisspg/170/base 2025-12-04T09:21:04.0664241Z * [new branch] gh/drisspg/170/head -> origin/gh/drisspg/170/head 2025-12-04T09:21:04.0665428Z * [new branch] gh/drisspg/170/orig -> origin/gh/drisspg/170/orig 2025-12-04T09:21:04.0667408Z * [new branch] gh/drisspg/182/base -> origin/gh/drisspg/182/base 2025-12-04T09:21:04.0668743Z * [new branch] gh/drisspg/182/head -> origin/gh/drisspg/182/head 2025-12-04T09:21:04.0670367Z * [new branch] gh/drisspg/183/base -> origin/gh/drisspg/183/base 2025-12-04T09:21:04.0671482Z * [new branch] gh/drisspg/183/head -> origin/gh/drisspg/183/head 2025-12-04T09:21:04.0672996Z * [new branch] gh/drisspg/184/base -> origin/gh/drisspg/184/base 2025-12-04T09:21:04.0674171Z * [new branch] gh/drisspg/184/head -> origin/gh/drisspg/184/head 2025-12-04T09:21:04.0675864Z * [new branch] gh/drisspg/185/base -> origin/gh/drisspg/185/base 2025-12-04T09:21:04.0677110Z * [new branch] gh/drisspg/185/head -> origin/gh/drisspg/185/head 2025-12-04T09:21:04.0678844Z * [new branch] gh/drisspg/194/base -> origin/gh/drisspg/194/base 2025-12-04T09:21:04.0680067Z * [new branch] gh/drisspg/194/head -> origin/gh/drisspg/194/head 2025-12-04T09:21:04.0681253Z * [new branch] gh/drisspg/194/orig -> origin/gh/drisspg/194/orig 2025-12-04T09:21:04.0682869Z * [new branch] gh/drisspg/200/base -> origin/gh/drisspg/200/base 2025-12-04T09:21:04.0684025Z * [new branch] gh/drisspg/200/head -> origin/gh/drisspg/200/head 2025-12-04T09:21:04.0685260Z * [new branch] gh/drisspg/200/orig -> origin/gh/drisspg/200/orig 2025-12-04T09:21:04.0686894Z * [new branch] gh/drisspg/218/base -> origin/gh/drisspg/218/base 2025-12-04T09:21:04.0688061Z * [new branch] gh/drisspg/218/head -> origin/gh/drisspg/218/head 2025-12-04T09:21:04.0689262Z * [new branch] gh/drisspg/218/orig -> origin/gh/drisspg/218/orig 2025-12-04T09:21:04.0690844Z * [new branch] gh/drisspg/219/base -> origin/gh/drisspg/219/base 2025-12-04T09:21:04.0692059Z * [new branch] gh/drisspg/219/head -> origin/gh/drisspg/219/head 2025-12-04T09:21:04.0693284Z * [new branch] gh/drisspg/219/orig -> origin/gh/drisspg/219/orig 2025-12-04T09:21:04.0694822Z * [new branch] gh/drisspg/220/base -> origin/gh/drisspg/220/base 2025-12-04T09:21:04.0696064Z * [new branch] gh/drisspg/220/head -> origin/gh/drisspg/220/head 2025-12-04T09:21:04.0697251Z * [new branch] gh/drisspg/220/orig -> origin/gh/drisspg/220/orig 2025-12-04T09:21:04.0698839Z * [new branch] gh/drisspg/221/base -> origin/gh/drisspg/221/base 2025-12-04T09:21:04.0700041Z * [new branch] gh/drisspg/221/head -> origin/gh/drisspg/221/head 2025-12-04T09:21:04.0701921Z * [new branch] gh/drisspg/221/orig -> origin/gh/drisspg/221/orig 2025-12-04T09:21:04.0703774Z * [new branch] gh/drisspg/222/base -> origin/gh/drisspg/222/base 2025-12-04T09:21:04.0705011Z * [new branch] gh/drisspg/222/head -> origin/gh/drisspg/222/head 2025-12-04T09:21:04.0706276Z * [new branch] gh/drisspg/222/orig -> origin/gh/drisspg/222/orig 2025-12-04T09:21:04.0708006Z * [new branch] gh/drisspg/223/base -> origin/gh/drisspg/223/base 2025-12-04T09:21:04.0709232Z * [new branch] gh/drisspg/223/head -> origin/gh/drisspg/223/head 2025-12-04T09:21:04.0710481Z * [new branch] gh/drisspg/223/orig -> origin/gh/drisspg/223/orig 2025-12-04T09:21:04.0712074Z * [new branch] gh/drisspg/224/base -> origin/gh/drisspg/224/base 2025-12-04T09:21:04.0718415Z * [new branch] gh/drisspg/224/head -> origin/gh/drisspg/224/head 2025-12-04T09:21:04.0719630Z * [new branch] gh/drisspg/224/orig -> origin/gh/drisspg/224/orig 2025-12-04T09:21:04.0721297Z * [new branch] gh/drisspg/225/base -> origin/gh/drisspg/225/base 2025-12-04T09:21:04.0722512Z * [new branch] gh/drisspg/225/head -> origin/gh/drisspg/225/head 2025-12-04T09:21:04.0723779Z * [new branch] gh/drisspg/225/orig -> origin/gh/drisspg/225/orig 2025-12-04T09:21:04.0725609Z * [new branch] gh/drisspg/226/base -> origin/gh/drisspg/226/base 2025-12-04T09:21:04.0726848Z * [new branch] gh/drisspg/226/head -> origin/gh/drisspg/226/head 2025-12-04T09:21:04.0728069Z * [new branch] gh/drisspg/226/orig -> origin/gh/drisspg/226/orig 2025-12-04T09:21:04.0730073Z * [new branch] gh/drisspg/227/base -> origin/gh/drisspg/227/base 2025-12-04T09:21:04.0731303Z * [new branch] gh/drisspg/227/head -> origin/gh/drisspg/227/head 2025-12-04T09:21:04.0732425Z * [new branch] gh/drisspg/227/orig -> origin/gh/drisspg/227/orig 2025-12-04T09:21:04.0734042Z * [new branch] gh/drisspg/228/base -> origin/gh/drisspg/228/base 2025-12-04T09:21:04.0735274Z * [new branch] gh/drisspg/228/head -> origin/gh/drisspg/228/head 2025-12-04T09:21:04.0736457Z * [new branch] gh/drisspg/228/orig -> origin/gh/drisspg/228/orig 2025-12-04T09:21:04.0738035Z * [new branch] gh/drisspg/229/base -> origin/gh/drisspg/229/base 2025-12-04T09:21:04.0739229Z * [new branch] gh/drisspg/229/head -> origin/gh/drisspg/229/head 2025-12-04T09:21:04.0740506Z * [new branch] gh/drisspg/229/orig -> origin/gh/drisspg/229/orig 2025-12-04T09:21:04.0742175Z * [new branch] gh/drisspg/230/base -> origin/gh/drisspg/230/base 2025-12-04T09:21:04.0743356Z * [new branch] gh/drisspg/230/head -> origin/gh/drisspg/230/head 2025-12-04T09:21:04.0744551Z * [new branch] gh/drisspg/230/orig -> origin/gh/drisspg/230/orig 2025-12-04T09:21:04.0746602Z * [new branch] gh/dsjohns2/1/base -> origin/gh/dsjohns2/1/base 2025-12-04T09:21:04.0748179Z * [new branch] gh/dsjohns2/1/head -> origin/gh/dsjohns2/1/head 2025-12-04T09:21:04.0750182Z * [new branch] gh/dzmitry-huba/1/base -> origin/gh/dzmitry-huba/1/base 2025-12-04T09:21:04.0751473Z * [new branch] gh/dzmitry-huba/1/head -> origin/gh/dzmitry-huba/1/head 2025-12-04T09:21:04.0753366Z * [new branch] gh/dzmitry-huba/12/base -> origin/gh/dzmitry-huba/12/base 2025-12-04T09:21:04.0754700Z * [new branch] gh/dzmitry-huba/12/head -> origin/gh/dzmitry-huba/12/head 2025-12-04T09:21:04.0756021Z * [new branch] gh/dzmitry-huba/12/orig -> origin/gh/dzmitry-huba/12/orig 2025-12-04T09:21:04.0757871Z * [new branch] gh/dzmitry-huba/13/base -> origin/gh/dzmitry-huba/13/base 2025-12-04T09:21:04.0759299Z * [new branch] gh/dzmitry-huba/13/head -> origin/gh/dzmitry-huba/13/head 2025-12-04T09:21:04.0760479Z * [new branch] gh/dzmitry-huba/13/orig -> origin/gh/dzmitry-huba/13/orig 2025-12-04T09:21:04.0762073Z * [new branch] gh/dzmitry-huba/14/base -> origin/gh/dzmitry-huba/14/base 2025-12-04T09:21:04.0763274Z * [new branch] gh/dzmitry-huba/14/head -> origin/gh/dzmitry-huba/14/head 2025-12-04T09:21:04.0764477Z * [new branch] gh/dzmitry-huba/14/orig -> origin/gh/dzmitry-huba/14/orig 2025-12-04T09:21:04.0766366Z * [new branch] gh/dzmitry-huba/15/base -> origin/gh/dzmitry-huba/15/base 2025-12-04T09:21:04.0767590Z * [new branch] gh/dzmitry-huba/15/head -> origin/gh/dzmitry-huba/15/head 2025-12-04T09:21:04.0768746Z * [new branch] gh/dzmitry-huba/15/orig -> origin/gh/dzmitry-huba/15/orig 2025-12-04T09:21:04.0770537Z * [new branch] gh/dzmitry-huba/16/base -> origin/gh/dzmitry-huba/16/base 2025-12-04T09:21:04.0771867Z * [new branch] gh/dzmitry-huba/16/head -> origin/gh/dzmitry-huba/16/head 2025-12-04T09:21:04.0773077Z * [new branch] gh/dzmitry-huba/16/orig -> origin/gh/dzmitry-huba/16/orig 2025-12-04T09:21:04.0774884Z * [new branch] gh/dzmitry-huba/17/base -> origin/gh/dzmitry-huba/17/base 2025-12-04T09:21:04.0780355Z * [new branch] gh/dzmitry-huba/17/head -> origin/gh/dzmitry-huba/17/head 2025-12-04T09:21:04.0780641Z * [new branch] gh/dzmitry-huba/17/orig -> origin/gh/dzmitry-huba/17/orig 2025-12-04T09:21:04.0780934Z * [new branch] gh/dzmitry-huba/2/base -> origin/gh/dzmitry-huba/2/base 2025-12-04T09:21:04.0781209Z * [new branch] gh/dzmitry-huba/2/head -> origin/gh/dzmitry-huba/2/head 2025-12-04T09:21:04.0781935Z * [new branch] gh/dzmitry-huba/3/base -> origin/gh/dzmitry-huba/3/base 2025-12-04T09:21:04.0783179Z * [new branch] gh/dzmitry-huba/3/head -> origin/gh/dzmitry-huba/3/head 2025-12-04T09:21:04.0785242Z * [new branch] gh/eellison/808/base -> origin/gh/eellison/808/base 2025-12-04T09:21:04.0786632Z * [new branch] gh/eellison/808/head -> origin/gh/eellison/808/head 2025-12-04T09:21:04.0787947Z * [new branch] gh/eellison/808/orig -> origin/gh/eellison/808/orig 2025-12-04T09:21:04.0790037Z * [new branch] gh/eellison/822/base -> origin/gh/eellison/822/base 2025-12-04T09:21:04.0791361Z * [new branch] gh/eellison/822/head -> origin/gh/eellison/822/head 2025-12-04T09:21:04.0792636Z * [new branch] gh/eellison/822/orig -> origin/gh/eellison/822/orig 2025-12-04T09:21:04.0794285Z * [new branch] gh/eellison/823/base -> origin/gh/eellison/823/base 2025-12-04T09:21:04.0795554Z * [new branch] gh/eellison/823/head -> origin/gh/eellison/823/head 2025-12-04T09:21:04.0796797Z * [new branch] gh/eellison/823/orig -> origin/gh/eellison/823/orig 2025-12-04T09:21:04.0798406Z * [new branch] gh/eellison/862/base -> origin/gh/eellison/862/base 2025-12-04T09:21:04.0799802Z * [new branch] gh/eellison/862/head -> origin/gh/eellison/862/head 2025-12-04T09:21:04.0801440Z * [new branch] gh/eellison/862/orig -> origin/gh/eellison/862/orig 2025-12-04T09:21:04.0803453Z * [new branch] gh/eellison/863/base -> origin/gh/eellison/863/base 2025-12-04T09:21:04.0804679Z * [new branch] gh/eellison/863/head -> origin/gh/eellison/863/head 2025-12-04T09:21:04.0805960Z * [new branch] gh/eellison/863/orig -> origin/gh/eellison/863/orig 2025-12-04T09:21:04.0807589Z * [new branch] gh/eellison/864/base -> origin/gh/eellison/864/base 2025-12-04T09:21:04.0808853Z * [new branch] gh/eellison/864/head -> origin/gh/eellison/864/head 2025-12-04T09:21:04.0810128Z * [new branch] gh/eellison/864/orig -> origin/gh/eellison/864/orig 2025-12-04T09:21:04.0811755Z * [new branch] gh/eellison/865/base -> origin/gh/eellison/865/base 2025-12-04T09:21:04.0812988Z * [new branch] gh/eellison/865/head -> origin/gh/eellison/865/head 2025-12-04T09:21:04.0814309Z * [new branch] gh/eellison/865/orig -> origin/gh/eellison/865/orig 2025-12-04T09:21:04.0815898Z * [new branch] gh/eellison/866/base -> origin/gh/eellison/866/base 2025-12-04T09:21:04.0817112Z * [new branch] gh/eellison/866/head -> origin/gh/eellison/866/head 2025-12-04T09:21:04.0818276Z * [new branch] gh/eellison/866/orig -> origin/gh/eellison/866/orig 2025-12-04T09:21:04.0820065Z * [new branch] gh/eellison/867/base -> origin/gh/eellison/867/base 2025-12-04T09:21:04.0821185Z * [new branch] gh/eellison/867/head -> origin/gh/eellison/867/head 2025-12-04T09:21:04.0822382Z * [new branch] gh/eellison/867/orig -> origin/gh/eellison/867/orig 2025-12-04T09:21:04.0824365Z * [new branch] gh/eellison/868/base -> origin/gh/eellison/868/base 2025-12-04T09:21:04.0825857Z * [new branch] gh/eellison/868/head -> origin/gh/eellison/868/head 2025-12-04T09:21:04.0827394Z * [new branch] gh/eellison/868/orig -> origin/gh/eellison/868/orig 2025-12-04T09:21:04.0829117Z * [new branch] gh/eellison/869/base -> origin/gh/eellison/869/base 2025-12-04T09:21:04.0830293Z * [new branch] gh/eellison/869/head -> origin/gh/eellison/869/head 2025-12-04T09:21:04.0831475Z * [new branch] gh/eellison/869/orig -> origin/gh/eellison/869/orig 2025-12-04T09:21:04.0833163Z * [new branch] gh/eellison/870/base -> origin/gh/eellison/870/base 2025-12-04T09:21:04.0834348Z * [new branch] gh/eellison/870/head -> origin/gh/eellison/870/head 2025-12-04T09:21:04.0835536Z * [new branch] gh/eellison/870/orig -> origin/gh/eellison/870/orig 2025-12-04T09:21:04.0837459Z * [new branch] gh/eellison/871/base -> origin/gh/eellison/871/base 2025-12-04T09:21:04.0842987Z * [new branch] gh/eellison/871/head -> origin/gh/eellison/871/head 2025-12-04T09:21:04.0843276Z * [new branch] gh/eellison/871/orig -> origin/gh/eellison/871/orig 2025-12-04T09:21:04.0843738Z * [new branch] gh/eellison/872/base -> origin/gh/eellison/872/base 2025-12-04T09:21:04.0844014Z * [new branch] gh/eellison/872/head -> origin/gh/eellison/872/head 2025-12-04T09:21:04.0844380Z * [new branch] gh/eellison/872/orig -> origin/gh/eellison/872/orig 2025-12-04T09:21:04.0846257Z * [new branch] gh/eellison/873/base -> origin/gh/eellison/873/base 2025-12-04T09:21:04.0847458Z * [new branch] gh/eellison/873/head -> origin/gh/eellison/873/head 2025-12-04T09:21:04.0848692Z * [new branch] gh/eellison/873/orig -> origin/gh/eellison/873/orig 2025-12-04T09:21:04.0850459Z * [new branch] gh/eellison/874/base -> origin/gh/eellison/874/base 2025-12-04T09:21:04.0851672Z * [new branch] gh/eellison/874/head -> origin/gh/eellison/874/head 2025-12-04T09:21:04.0852866Z * [new branch] gh/eellison/874/orig -> origin/gh/eellison/874/orig 2025-12-04T09:21:04.0854983Z * [new branch] gh/eellison/875/base -> origin/gh/eellison/875/base 2025-12-04T09:21:04.0856302Z * [new branch] gh/eellison/875/head -> origin/gh/eellison/875/head 2025-12-04T09:21:04.0857525Z * [new branch] gh/eellison/875/orig -> origin/gh/eellison/875/orig 2025-12-04T09:21:04.0859341Z * [new branch] gh/eellison/876/base -> origin/gh/eellison/876/base 2025-12-04T09:21:04.0860559Z * [new branch] gh/eellison/876/head -> origin/gh/eellison/876/head 2025-12-04T09:21:04.0861762Z * [new branch] gh/eellison/876/orig -> origin/gh/eellison/876/orig 2025-12-04T09:21:04.0863429Z * [new branch] gh/eellison/877/base -> origin/gh/eellison/877/base 2025-12-04T09:21:04.0864805Z * [new branch] gh/eellison/877/head -> origin/gh/eellison/877/head 2025-12-04T09:21:04.0865816Z * [new branch] gh/eellison/877/orig -> origin/gh/eellison/877/orig 2025-12-04T09:21:04.0867911Z * [new branch] gh/eellison/878/base -> origin/gh/eellison/878/base 2025-12-04T09:21:04.0869084Z * [new branch] gh/eellison/878/head -> origin/gh/eellison/878/head 2025-12-04T09:21:04.0870523Z * [new branch] gh/eellison/878/orig -> origin/gh/eellison/878/orig 2025-12-04T09:21:04.0872359Z * [new branch] gh/eellison/879/base -> origin/gh/eellison/879/base 2025-12-04T09:21:04.0873641Z * [new branch] gh/eellison/879/head -> origin/gh/eellison/879/head 2025-12-04T09:21:04.0874857Z * [new branch] gh/eellison/879/orig -> origin/gh/eellison/879/orig 2025-12-04T09:21:04.0876527Z * [new branch] gh/eellison/880/base -> origin/gh/eellison/880/base 2025-12-04T09:21:04.0877807Z * [new branch] gh/eellison/880/head -> origin/gh/eellison/880/head 2025-12-04T09:21:04.0879090Z * [new branch] gh/eellison/880/orig -> origin/gh/eellison/880/orig 2025-12-04T09:21:04.0880904Z * [new branch] gh/eellison/881/base -> origin/gh/eellison/881/base 2025-12-04T09:21:04.0882186Z * [new branch] gh/eellison/881/head -> origin/gh/eellison/881/head 2025-12-04T09:21:04.0883372Z * [new branch] gh/eellison/881/orig -> origin/gh/eellison/881/orig 2025-12-04T09:21:04.0884945Z * [new branch] gh/eellison/882/base -> origin/gh/eellison/882/base 2025-12-04T09:21:04.0886150Z * [new branch] gh/eellison/882/head -> origin/gh/eellison/882/head 2025-12-04T09:21:04.0887538Z * [new branch] gh/eellison/882/orig -> origin/gh/eellison/882/orig 2025-12-04T09:21:04.0889058Z * [new branch] gh/eellison/883/base -> origin/gh/eellison/883/base 2025-12-04T09:21:04.0890274Z * [new branch] gh/eellison/883/head -> origin/gh/eellison/883/head 2025-12-04T09:21:04.0891542Z * [new branch] gh/eellison/883/orig -> origin/gh/eellison/883/orig 2025-12-04T09:21:04.0893026Z * [new branch] gh/eellison/884/base -> origin/gh/eellison/884/base 2025-12-04T09:21:04.0894244Z * [new branch] gh/eellison/884/head -> origin/gh/eellison/884/head 2025-12-04T09:21:04.0895382Z * [new branch] gh/eellison/884/orig -> origin/gh/eellison/884/orig 2025-12-04T09:21:04.0897325Z * [new branch] gh/etaf/147/base -> origin/gh/etaf/147/base 2025-12-04T09:21:04.0898607Z * [new branch] gh/etaf/147/head -> origin/gh/etaf/147/head 2025-12-04T09:21:04.0906169Z * [new branch] gh/etaf/154/base -> origin/gh/etaf/154/base 2025-12-04T09:21:04.0907691Z * [new branch] gh/etaf/154/head -> origin/gh/etaf/154/head 2025-12-04T09:21:04.0908942Z * [new branch] gh/etaf/154/orig -> origin/gh/etaf/154/orig 2025-12-04T09:21:04.0910572Z * [new branch] gh/etaf/156/base -> origin/gh/etaf/156/base 2025-12-04T09:21:04.0911804Z * [new branch] gh/etaf/156/head -> origin/gh/etaf/156/head 2025-12-04T09:21:04.0913061Z * [new branch] gh/etaf/156/orig -> origin/gh/etaf/156/orig 2025-12-04T09:21:04.0915029Z * [new branch] gh/etaf/157/base -> origin/gh/etaf/157/base 2025-12-04T09:21:04.0916310Z * [new branch] gh/etaf/157/head -> origin/gh/etaf/157/head 2025-12-04T09:21:04.0917594Z * [new branch] gh/etaf/157/orig -> origin/gh/etaf/157/orig 2025-12-04T09:21:04.0919708Z * [new branch] gh/etaf/158/base -> origin/gh/etaf/158/base 2025-12-04T09:21:04.0920506Z * [new branch] gh/etaf/158/head -> origin/gh/etaf/158/head 2025-12-04T09:21:04.0921776Z * [new branch] gh/etaf/158/orig -> origin/gh/etaf/158/orig 2025-12-04T09:21:04.0923406Z * [new branch] gh/etaf/159/base -> origin/gh/etaf/159/base 2025-12-04T09:21:04.0924680Z * [new branch] gh/etaf/159/head -> origin/gh/etaf/159/head 2025-12-04T09:21:04.0925876Z * [new branch] gh/etaf/159/orig -> origin/gh/etaf/159/orig 2025-12-04T09:21:04.0927531Z * [new branch] gh/etaf/160/base -> origin/gh/etaf/160/base 2025-12-04T09:21:04.0928834Z * [new branch] gh/etaf/160/head -> origin/gh/etaf/160/head 2025-12-04T09:21:04.0930023Z * [new branch] gh/etaf/160/orig -> origin/gh/etaf/160/orig 2025-12-04T09:21:04.0931822Z * [new branch] gh/etaf/161/base -> origin/gh/etaf/161/base 2025-12-04T09:21:04.0933090Z * [new branch] gh/etaf/161/head -> origin/gh/etaf/161/head 2025-12-04T09:21:04.0934315Z * [new branch] gh/etaf/161/orig -> origin/gh/etaf/161/orig 2025-12-04T09:21:04.0935958Z * [new branch] gh/etaf/166/base -> origin/gh/etaf/166/base 2025-12-04T09:21:04.0937306Z * [new branch] gh/etaf/166/head -> origin/gh/etaf/166/head 2025-12-04T09:21:04.0938547Z * [new branch] gh/etaf/166/orig -> origin/gh/etaf/166/orig 2025-12-04T09:21:04.0940052Z * [new branch] gh/etaf/167/base -> origin/gh/etaf/167/base 2025-12-04T09:21:04.0941275Z * [new branch] gh/etaf/167/head -> origin/gh/etaf/167/head 2025-12-04T09:21:04.0942447Z * [new branch] gh/etaf/167/orig -> origin/gh/etaf/167/orig 2025-12-04T09:21:04.0944170Z * [new branch] gh/etaf/168/base -> origin/gh/etaf/168/base 2025-12-04T09:21:04.0945418Z * [new branch] gh/etaf/168/head -> origin/gh/etaf/168/head 2025-12-04T09:21:04.0946712Z * [new branch] gh/etaf/168/orig -> origin/gh/etaf/168/orig 2025-12-04T09:21:04.0948924Z * [new branch] gh/etaf/172/base -> origin/gh/etaf/172/base 2025-12-04T09:21:04.0950106Z * [new branch] gh/etaf/172/head -> origin/gh/etaf/172/head 2025-12-04T09:21:04.0951411Z * [new branch] gh/etaf/172/orig -> origin/gh/etaf/172/orig 2025-12-04T09:21:04.0953206Z * [new branch] gh/etaf/173/base -> origin/gh/etaf/173/base 2025-12-04T09:21:04.0954579Z * [new branch] gh/etaf/173/head -> origin/gh/etaf/173/head 2025-12-04T09:21:04.0955866Z * [new branch] gh/etaf/173/orig -> origin/gh/etaf/173/orig 2025-12-04T09:21:04.0957578Z * [new branch] gh/etaf/174/base -> origin/gh/etaf/174/base 2025-12-04T09:21:04.0958942Z * [new branch] gh/etaf/174/head -> origin/gh/etaf/174/head 2025-12-04T09:21:04.0960581Z * [new branch] gh/etaf/175/base -> origin/gh/etaf/175/base 2025-12-04T09:21:04.0961775Z * [new branch] gh/etaf/175/head -> origin/gh/etaf/175/head 2025-12-04T09:21:04.0968012Z * [new branch] gh/etaf/175/orig -> origin/gh/etaf/175/orig 2025-12-04T09:21:04.0969797Z * [new branch] gh/etaf/176/base -> origin/gh/etaf/176/base 2025-12-04T09:21:04.0971285Z * [new branch] gh/etaf/176/head -> origin/gh/etaf/176/head 2025-12-04T09:21:04.0972558Z * [new branch] gh/etaf/176/orig -> origin/gh/etaf/176/orig 2025-12-04T09:21:04.0974630Z * [new branch] gh/etaf/177/base -> origin/gh/etaf/177/base 2025-12-04T09:21:04.0976083Z * [new branch] gh/etaf/177/head -> origin/gh/etaf/177/head 2025-12-04T09:21:04.0977339Z * [new branch] gh/etaf/177/orig -> origin/gh/etaf/177/orig 2025-12-04T09:21:04.0979259Z * [new branch] gh/etaf/178/base -> origin/gh/etaf/178/base 2025-12-04T09:21:04.0980653Z * [new branch] gh/etaf/178/head -> origin/gh/etaf/178/head 2025-12-04T09:21:04.0981930Z * [new branch] gh/etaf/178/orig -> origin/gh/etaf/178/orig 2025-12-04T09:21:04.0983610Z * [new branch] gh/etaf/179/base -> origin/gh/etaf/179/base 2025-12-04T09:21:04.0984837Z * [new branch] gh/etaf/179/head -> origin/gh/etaf/179/head 2025-12-04T09:21:04.0986062Z * [new branch] gh/etaf/179/orig -> origin/gh/etaf/179/orig 2025-12-04T09:21:04.0987931Z * [new branch] gh/etaf/180/base -> origin/gh/etaf/180/base 2025-12-04T09:21:04.0989456Z * [new branch] gh/etaf/180/head -> origin/gh/etaf/180/head 2025-12-04T09:21:04.0990712Z * [new branch] gh/etaf/180/orig -> origin/gh/etaf/180/orig 2025-12-04T09:21:04.0992770Z * [new branch] gh/exclamaforte/1/base -> origin/gh/exclamaforte/1/base 2025-12-04T09:21:04.0994026Z * [new branch] gh/exclamaforte/1/head -> origin/gh/exclamaforte/1/head 2025-12-04T09:21:04.0995622Z * [new branch] gh/exclamaforte/2/base -> origin/gh/exclamaforte/2/base 2025-12-04T09:21:04.0996786Z * [new branch] gh/exclamaforte/2/head -> origin/gh/exclamaforte/2/head 2025-12-04T09:21:04.0998397Z * [new branch] gh/exclamaforte/3/base -> origin/gh/exclamaforte/3/base 2025-12-04T09:21:04.0999774Z * [new branch] gh/exclamaforte/3/head -> origin/gh/exclamaforte/3/head 2025-12-04T09:21:04.1002162Z * [new branch] gh/exclamaforte/4/base -> origin/gh/exclamaforte/4/base 2025-12-04T09:21:04.1003399Z * [new branch] gh/exclamaforte/4/head -> origin/gh/exclamaforte/4/head 2025-12-04T09:21:04.1005486Z * [new branch] gh/ezyang/2374/base -> origin/gh/ezyang/2374/base 2025-12-04T09:21:04.1006788Z * [new branch] gh/ezyang/2374/head -> origin/gh/ezyang/2374/head 2025-12-04T09:21:04.1008106Z * [new branch] gh/ezyang/2374/orig -> origin/gh/ezyang/2374/orig 2025-12-04T09:21:04.1009648Z * [new branch] gh/ezyang/2973/base -> origin/gh/ezyang/2973/base 2025-12-04T09:21:04.1010873Z * [new branch] gh/ezyang/2973/head -> origin/gh/ezyang/2973/head 2025-12-04T09:21:04.1012141Z * [new branch] gh/ezyang/2973/orig -> origin/gh/ezyang/2973/orig 2025-12-04T09:21:04.1013911Z * [new branch] gh/ezyang/2974/base -> origin/gh/ezyang/2974/base 2025-12-04T09:21:04.1015101Z * [new branch] gh/ezyang/2974/head -> origin/gh/ezyang/2974/head 2025-12-04T09:21:04.1016374Z * [new branch] gh/ezyang/2974/orig -> origin/gh/ezyang/2974/orig 2025-12-04T09:21:04.1017932Z * [new branch] gh/ezyang/3131/base -> origin/gh/ezyang/3131/base 2025-12-04T09:21:04.1019108Z * [new branch] gh/ezyang/3131/head -> origin/gh/ezyang/3131/head 2025-12-04T09:21:04.1020334Z * [new branch] gh/ezyang/3131/orig -> origin/gh/ezyang/3131/orig 2025-12-04T09:21:04.1022045Z * [new branch] gh/ezyang/3139/base -> origin/gh/ezyang/3139/base 2025-12-04T09:21:04.1023210Z * [new branch] gh/ezyang/3139/head -> origin/gh/ezyang/3139/head 2025-12-04T09:21:04.1024393Z * [new branch] gh/ezyang/3139/orig -> origin/gh/ezyang/3139/orig 2025-12-04T09:21:04.1030350Z * [new branch] gh/ezyang/3140/base -> origin/gh/ezyang/3140/base 2025-12-04T09:21:04.1030604Z * [new branch] gh/ezyang/3140/head -> origin/gh/ezyang/3140/head 2025-12-04T09:21:04.1030873Z * [new branch] gh/ezyang/3140/orig -> origin/gh/ezyang/3140/orig 2025-12-04T09:21:04.1031142Z * [new branch] gh/ezyang/3143/base -> origin/gh/ezyang/3143/base 2025-12-04T09:21:04.1032204Z * [new branch] gh/ezyang/3143/head -> origin/gh/ezyang/3143/head 2025-12-04T09:21:04.1033423Z * [new branch] gh/ezyang/3143/orig -> origin/gh/ezyang/3143/orig 2025-12-04T09:21:04.1035079Z * [new branch] gh/ezyang/3144/base -> origin/gh/ezyang/3144/base 2025-12-04T09:21:04.1036380Z * [new branch] gh/ezyang/3144/head -> origin/gh/ezyang/3144/head 2025-12-04T09:21:04.1037572Z * [new branch] gh/ezyang/3144/orig -> origin/gh/ezyang/3144/orig 2025-12-04T09:21:04.1039410Z * [new branch] gh/ezyang/3167/base -> origin/gh/ezyang/3167/base 2025-12-04T09:21:04.1040591Z * [new branch] gh/ezyang/3167/head -> origin/gh/ezyang/3167/head 2025-12-04T09:21:04.1041784Z * [new branch] gh/ezyang/3167/orig -> origin/gh/ezyang/3167/orig 2025-12-04T09:21:04.1043381Z * [new branch] gh/ezyang/3173/base -> origin/gh/ezyang/3173/base 2025-12-04T09:21:04.1044551Z * [new branch] gh/ezyang/3173/head -> origin/gh/ezyang/3173/head 2025-12-04T09:21:04.1045847Z * [new branch] gh/ezyang/3173/orig -> origin/gh/ezyang/3173/orig 2025-12-04T09:21:04.1047404Z * [new branch] gh/ezyang/3175/base -> origin/gh/ezyang/3175/base 2025-12-04T09:21:04.1048630Z * [new branch] gh/ezyang/3175/head -> origin/gh/ezyang/3175/head 2025-12-04T09:21:04.1049774Z * [new branch] gh/ezyang/3175/orig -> origin/gh/ezyang/3175/orig 2025-12-04T09:21:04.1051338Z * [new branch] gh/ezyang/3182/base -> origin/gh/ezyang/3182/base 2025-12-04T09:21:04.1052686Z * [new branch] gh/ezyang/3182/head -> origin/gh/ezyang/3182/head 2025-12-04T09:21:04.1053710Z * [new branch] gh/ezyang/3182/orig -> origin/gh/ezyang/3182/orig 2025-12-04T09:21:04.1055381Z * [new branch] gh/ezyang/3185/base -> origin/gh/ezyang/3185/base 2025-12-04T09:21:04.1056823Z * [new branch] gh/ezyang/3185/head -> origin/gh/ezyang/3185/head 2025-12-04T09:21:04.1057796Z * [new branch] gh/ezyang/3185/orig -> origin/gh/ezyang/3185/orig 2025-12-04T09:21:04.1059363Z * [new branch] gh/ezyang/3189/base -> origin/gh/ezyang/3189/base 2025-12-04T09:21:04.1060546Z * [new branch] gh/ezyang/3189/head -> origin/gh/ezyang/3189/head 2025-12-04T09:21:04.1061780Z * [new branch] gh/ezyang/3189/orig -> origin/gh/ezyang/3189/orig 2025-12-04T09:21:04.1063319Z * [new branch] gh/ezyang/3191/base -> origin/gh/ezyang/3191/base 2025-12-04T09:21:04.1064539Z * [new branch] gh/ezyang/3191/head -> origin/gh/ezyang/3191/head 2025-12-04T09:21:04.1065726Z * [new branch] gh/ezyang/3191/orig -> origin/gh/ezyang/3191/orig 2025-12-04T09:21:04.1068188Z * [new branch] gh/ezyang/3192/base -> origin/gh/ezyang/3192/base 2025-12-04T09:21:04.1069432Z * [new branch] gh/ezyang/3192/head -> origin/gh/ezyang/3192/head 2025-12-04T09:21:04.1070805Z * [new branch] gh/ezyang/3192/orig -> origin/gh/ezyang/3192/orig 2025-12-04T09:21:04.1072606Z * [new branch] gh/ezyang/3193/base -> origin/gh/ezyang/3193/base 2025-12-04T09:21:04.1073809Z * [new branch] gh/ezyang/3193/head -> origin/gh/ezyang/3193/head 2025-12-04T09:21:04.1075067Z * [new branch] gh/ezyang/3193/orig -> origin/gh/ezyang/3193/orig 2025-12-04T09:21:04.1076828Z * [new branch] gh/ezyang/3194/base -> origin/gh/ezyang/3194/base 2025-12-04T09:21:04.1078078Z * [new branch] gh/ezyang/3194/head -> origin/gh/ezyang/3194/head 2025-12-04T09:21:04.1079427Z * [new branch] gh/ezyang/3194/orig -> origin/gh/ezyang/3194/orig 2025-12-04T09:21:04.1080998Z * [new branch] gh/ezyang/3195/base -> origin/gh/ezyang/3195/base 2025-12-04T09:21:04.1082228Z * [new branch] gh/ezyang/3195/head -> origin/gh/ezyang/3195/head 2025-12-04T09:21:04.1083381Z * [new branch] gh/ezyang/3195/orig -> origin/gh/ezyang/3195/orig 2025-12-04T09:21:04.1085016Z * [new branch] gh/ezyang/3196/base -> origin/gh/ezyang/3196/base 2025-12-04T09:21:04.1086212Z * [new branch] gh/ezyang/3196/head -> origin/gh/ezyang/3196/head 2025-12-04T09:21:04.1092496Z * [new branch] gh/ezyang/3196/orig -> origin/gh/ezyang/3196/orig 2025-12-04T09:21:04.1092827Z * [new branch] gh/ezyang/3197/base -> origin/gh/ezyang/3197/base 2025-12-04T09:21:04.1093096Z * [new branch] gh/ezyang/3197/head -> origin/gh/ezyang/3197/head 2025-12-04T09:21:04.1093350Z * [new branch] gh/ezyang/3197/orig -> origin/gh/ezyang/3197/orig 2025-12-04T09:21:04.1094195Z * [new branch] gh/ezyang/3198/base -> origin/gh/ezyang/3198/base 2025-12-04T09:21:04.1095518Z * [new branch] gh/ezyang/3198/head -> origin/gh/ezyang/3198/head 2025-12-04T09:21:04.1096807Z * [new branch] gh/ezyang/3198/orig -> origin/gh/ezyang/3198/orig 2025-12-04T09:21:04.1098483Z * [new branch] gh/ezyang/3199/base -> origin/gh/ezyang/3199/base 2025-12-04T09:21:04.1099717Z * [new branch] gh/ezyang/3199/head -> origin/gh/ezyang/3199/head 2025-12-04T09:21:04.1101355Z * [new branch] gh/ezyang/3199/orig -> origin/gh/ezyang/3199/orig 2025-12-04T09:21:04.1103322Z * [new branch] gh/ezyang/3200/base -> origin/gh/ezyang/3200/base 2025-12-04T09:21:04.1104567Z * [new branch] gh/ezyang/3200/head -> origin/gh/ezyang/3200/head 2025-12-04T09:21:04.1105850Z * [new branch] gh/ezyang/3200/orig -> origin/gh/ezyang/3200/orig 2025-12-04T09:21:04.1107832Z * [new branch] gh/ezyang/3201/base -> origin/gh/ezyang/3201/base 2025-12-04T09:21:04.1109164Z * [new branch] gh/ezyang/3201/head -> origin/gh/ezyang/3201/head 2025-12-04T09:21:04.1110351Z * [new branch] gh/ezyang/3201/orig -> origin/gh/ezyang/3201/orig 2025-12-04T09:21:04.1111966Z * [new branch] gh/ezyang/3202/base -> origin/gh/ezyang/3202/base 2025-12-04T09:21:04.1113182Z * [new branch] gh/ezyang/3202/head -> origin/gh/ezyang/3202/head 2025-12-04T09:21:04.1114390Z * [new branch] gh/ezyang/3202/orig -> origin/gh/ezyang/3202/orig 2025-12-04T09:21:04.1116129Z * [new branch] gh/ezyang/3203/base -> origin/gh/ezyang/3203/base 2025-12-04T09:21:04.1117351Z * [new branch] gh/ezyang/3203/head -> origin/gh/ezyang/3203/head 2025-12-04T09:21:04.1118838Z * [new branch] gh/ezyang/3203/orig -> origin/gh/ezyang/3203/orig 2025-12-04T09:21:04.1120546Z * [new branch] gh/ezyang/3204/base -> origin/gh/ezyang/3204/base 2025-12-04T09:21:04.1121675Z * [new branch] gh/ezyang/3204/head -> origin/gh/ezyang/3204/head 2025-12-04T09:21:04.1122885Z * [new branch] gh/ezyang/3204/orig -> origin/gh/ezyang/3204/orig 2025-12-04T09:21:04.1124632Z * [new branch] gh/ezyang/3205/base -> origin/gh/ezyang/3205/base 2025-12-04T09:21:04.1126341Z * [new branch] gh/ezyang/3205/head -> origin/gh/ezyang/3205/head 2025-12-04T09:21:04.1127577Z * [new branch] gh/ezyang/3205/orig -> origin/gh/ezyang/3205/orig 2025-12-04T09:21:04.1129203Z * [new branch] gh/ezyang/3206/base -> origin/gh/ezyang/3206/base 2025-12-04T09:21:04.1130426Z * [new branch] gh/ezyang/3206/head -> origin/gh/ezyang/3206/head 2025-12-04T09:21:04.1131609Z * [new branch] gh/ezyang/3206/orig -> origin/gh/ezyang/3206/orig 2025-12-04T09:21:04.1133216Z * [new branch] gh/ezyang/3207/base -> origin/gh/ezyang/3207/base 2025-12-04T09:21:04.1134414Z * [new branch] gh/ezyang/3207/head -> origin/gh/ezyang/3207/head 2025-12-04T09:21:04.1135613Z * [new branch] gh/ezyang/3207/orig -> origin/gh/ezyang/3207/orig 2025-12-04T09:21:04.1137287Z * [new branch] gh/ezyang/3208/base -> origin/gh/ezyang/3208/base 2025-12-04T09:21:04.1138484Z * [new branch] gh/ezyang/3208/head -> origin/gh/ezyang/3208/head 2025-12-04T09:21:04.1139740Z * [new branch] gh/ezyang/3208/orig -> origin/gh/ezyang/3208/orig 2025-12-04T09:21:04.1141438Z * [new branch] gh/ezyang/3209/base -> origin/gh/ezyang/3209/base 2025-12-04T09:21:04.1142674Z * [new branch] gh/ezyang/3209/head -> origin/gh/ezyang/3209/head 2025-12-04T09:21:04.1143886Z * [new branch] gh/ezyang/3209/orig -> origin/gh/ezyang/3209/orig 2025-12-04T09:21:04.1145770Z * [new branch] gh/fadara01/3/base -> origin/gh/fadara01/3/base 2025-12-04T09:21:04.1147252Z * [new branch] gh/fadara01/3/head -> origin/gh/fadara01/3/head 2025-12-04T09:21:04.1148583Z * [new branch] gh/fadara01/3/orig -> origin/gh/fadara01/3/orig 2025-12-04T09:21:04.1155451Z * [new branch] gh/fadara01/5/base -> origin/gh/fadara01/5/base 2025-12-04T09:21:04.1156792Z * [new branch] gh/fadara01/5/head -> origin/gh/fadara01/5/head 2025-12-04T09:21:04.1158005Z * [new branch] gh/fadara01/5/orig -> origin/gh/fadara01/5/orig 2025-12-04T09:21:04.1160146Z * [new branch] gh/fadara01/6/base -> origin/gh/fadara01/6/base 2025-12-04T09:21:04.1161708Z * [new branch] gh/fadara01/6/head -> origin/gh/fadara01/6/head 2025-12-04T09:21:04.1163238Z * [new branch] gh/fadara01/6/orig -> origin/gh/fadara01/6/orig 2025-12-04T09:21:04.1164933Z * [new branch] gh/fadara01/7/base -> origin/gh/fadara01/7/base 2025-12-04T09:21:04.1166059Z * [new branch] gh/fadara01/7/head -> origin/gh/fadara01/7/head 2025-12-04T09:21:04.1167446Z * [new branch] gh/fadara01/7/orig -> origin/gh/fadara01/7/orig 2025-12-04T09:21:04.1184619Z * [new branch] gh/fadara01/8/base -> origin/gh/fadara01/8/base 2025-12-04T09:21:04.1185296Z * [new branch] gh/fadara01/8/head -> origin/gh/fadara01/8/head 2025-12-04T09:21:04.1185898Z * [new branch] gh/fadara01/8/orig -> origin/gh/fadara01/8/orig 2025-12-04T09:21:04.1186583Z * [new branch] gh/fadara01/9/base -> origin/gh/fadara01/9/base 2025-12-04T09:21:04.1187236Z * [new branch] gh/fadara01/9/head -> origin/gh/fadara01/9/head 2025-12-04T09:21:04.1188078Z * [new branch] gh/fadara01/9/orig -> origin/gh/fadara01/9/orig 2025-12-04T09:21:04.1188708Z * [new branch] gh/fduwjj/182/base -> origin/gh/fduwjj/182/base 2025-12-04T09:21:04.1189422Z * [new branch] gh/fduwjj/182/head -> origin/gh/fduwjj/182/head 2025-12-04T09:21:04.1190066Z * [new branch] gh/fduwjj/182/orig -> origin/gh/fduwjj/182/orig 2025-12-04T09:21:04.1190701Z * [new branch] gh/fduwjj/211/base -> origin/gh/fduwjj/211/base 2025-12-04T09:21:04.1191323Z * [new branch] gh/fduwjj/211/head -> origin/gh/fduwjj/211/head 2025-12-04T09:21:04.1191997Z * [new branch] gh/fduwjj/211/orig -> origin/gh/fduwjj/211/orig 2025-12-04T09:21:04.1192644Z * [new branch] gh/fduwjj/212/base -> origin/gh/fduwjj/212/base 2025-12-04T09:21:04.1193298Z * [new branch] gh/fduwjj/212/head -> origin/gh/fduwjj/212/head 2025-12-04T09:21:04.1194952Z * [new branch] gh/fduwjj/212/orig -> origin/gh/fduwjj/212/orig 2025-12-04T09:21:04.1196672Z * [new branch] gh/fduwjj/213/base -> origin/gh/fduwjj/213/base 2025-12-04T09:21:04.1197918Z * [new branch] gh/fduwjj/213/head -> origin/gh/fduwjj/213/head 2025-12-04T09:21:04.1199163Z * [new branch] gh/fduwjj/213/orig -> origin/gh/fduwjj/213/orig 2025-12-04T09:21:04.1201390Z * [new branch] gh/fduwjj/226/base -> origin/gh/fduwjj/226/base 2025-12-04T09:21:04.1202908Z * [new branch] gh/fduwjj/226/head -> origin/gh/fduwjj/226/head 2025-12-04T09:21:04.1204068Z * [new branch] gh/fduwjj/226/orig -> origin/gh/fduwjj/226/orig 2025-12-04T09:21:04.1205979Z * [new branch] gh/fduwjj/229/base -> origin/gh/fduwjj/229/base 2025-12-04T09:21:04.1207097Z * [new branch] gh/fduwjj/229/head -> origin/gh/fduwjj/229/head 2025-12-04T09:21:04.1208345Z * [new branch] gh/fduwjj/229/orig -> origin/gh/fduwjj/229/orig 2025-12-04T09:21:04.1210079Z * [new branch] gh/fduwjj/233/base -> origin/gh/fduwjj/233/base 2025-12-04T09:21:04.1211297Z * [new branch] gh/fduwjj/233/head -> origin/gh/fduwjj/233/head 2025-12-04T09:21:04.1217139Z * [new branch] gh/fduwjj/233/orig -> origin/gh/fduwjj/233/orig 2025-12-04T09:21:04.1218880Z * [new branch] gh/fduwjj/234/base -> origin/gh/fduwjj/234/base 2025-12-04T09:21:04.1220298Z * [new branch] gh/fduwjj/234/head -> origin/gh/fduwjj/234/head 2025-12-04T09:21:04.1221601Z * [new branch] gh/fduwjj/234/orig -> origin/gh/fduwjj/234/orig 2025-12-04T09:21:04.1223354Z * [new branch] gh/fduwjj/235/base -> origin/gh/fduwjj/235/base 2025-12-04T09:21:04.1224874Z * [new branch] gh/fduwjj/235/head -> origin/gh/fduwjj/235/head 2025-12-04T09:21:04.1226043Z * [new branch] gh/fduwjj/235/orig -> origin/gh/fduwjj/235/orig 2025-12-04T09:21:04.1228024Z * [new branch] gh/fduwjj/236/base -> origin/gh/fduwjj/236/base 2025-12-04T09:21:04.1229121Z * [new branch] gh/fduwjj/236/head -> origin/gh/fduwjj/236/head 2025-12-04T09:21:04.1230347Z * [new branch] gh/fduwjj/236/orig -> origin/gh/fduwjj/236/orig 2025-12-04T09:21:04.1232713Z * [new branch] gh/fduwjj/237/base -> origin/gh/fduwjj/237/base 2025-12-04T09:21:04.1233927Z * [new branch] gh/fduwjj/237/head -> origin/gh/fduwjj/237/head 2025-12-04T09:21:04.1235203Z * [new branch] gh/fduwjj/237/orig -> origin/gh/fduwjj/237/orig 2025-12-04T09:21:04.1236947Z * [new branch] gh/fduwjj/238/base -> origin/gh/fduwjj/238/base 2025-12-04T09:21:04.1238156Z * [new branch] gh/fduwjj/238/head -> origin/gh/fduwjj/238/head 2025-12-04T09:21:04.1239503Z * [new branch] gh/fduwjj/238/orig -> origin/gh/fduwjj/238/orig 2025-12-04T09:21:04.1241248Z * [new branch] gh/fduwjj/239/base -> origin/gh/fduwjj/239/base 2025-12-04T09:21:04.1242629Z * [new branch] gh/fduwjj/239/head -> origin/gh/fduwjj/239/head 2025-12-04T09:21:04.1243741Z * [new branch] gh/fduwjj/239/orig -> origin/gh/fduwjj/239/orig 2025-12-04T09:21:04.1245661Z * [new branch] gh/fegin/332/base -> origin/gh/fegin/332/base 2025-12-04T09:21:04.1246804Z * [new branch] gh/fegin/332/head -> origin/gh/fegin/332/head 2025-12-04T09:21:04.1248048Z * [new branch] gh/fegin/332/orig -> origin/gh/fegin/332/orig 2025-12-04T09:21:04.1249686Z * [new branch] gh/fegin/333/base -> origin/gh/fegin/333/base 2025-12-04T09:21:04.1250835Z * [new branch] gh/fegin/333/head -> origin/gh/fegin/333/head 2025-12-04T09:21:04.1252120Z * [new branch] gh/fegin/333/orig -> origin/gh/fegin/333/orig 2025-12-04T09:21:04.1253702Z * [new branch] gh/fegin/334/base -> origin/gh/fegin/334/base 2025-12-04T09:21:04.1255244Z * [new branch] gh/fegin/334/head -> origin/gh/fegin/334/head 2025-12-04T09:21:04.1256647Z * [new branch] gh/fegin/334/orig -> origin/gh/fegin/334/orig 2025-12-04T09:21:04.1258318Z * [new branch] gh/fegin/335/base -> origin/gh/fegin/335/base 2025-12-04T09:21:04.1259513Z * [new branch] gh/fegin/335/head -> origin/gh/fegin/335/head 2025-12-04T09:21:04.1260768Z * [new branch] gh/fegin/335/orig -> origin/gh/fegin/335/orig 2025-12-04T09:21:04.1262655Z * [new branch] gh/fffrog/160/base -> origin/gh/fffrog/160/base 2025-12-04T09:21:04.1264295Z * [new branch] gh/fffrog/160/head -> origin/gh/fffrog/160/head 2025-12-04T09:21:04.1265913Z * [new branch] gh/fffrog/177/base -> origin/gh/fffrog/177/base 2025-12-04T09:21:04.1267365Z * [new branch] gh/fffrog/177/head -> origin/gh/fffrog/177/head 2025-12-04T09:21:04.1268810Z * [new branch] gh/fffrog/177/orig -> origin/gh/fffrog/177/orig 2025-12-04T09:21:04.1270448Z * [new branch] gh/fffrog/178/base -> origin/gh/fffrog/178/base 2025-12-04T09:21:04.1271630Z * [new branch] gh/fffrog/178/head -> origin/gh/fffrog/178/head 2025-12-04T09:21:04.1272844Z * [new branch] gh/fffrog/178/orig -> origin/gh/fffrog/178/orig 2025-12-04T09:21:04.1274630Z * [new branch] gh/fffrog/181/base -> origin/gh/fffrog/181/base 2025-12-04T09:21:04.1280889Z * [new branch] gh/fffrog/181/head -> origin/gh/fffrog/181/head 2025-12-04T09:21:04.1281508Z * [new branch] gh/fffrog/181/orig -> origin/gh/fffrog/181/orig 2025-12-04T09:21:04.1282208Z * [new branch] gh/fffrog/183/base -> origin/gh/fffrog/183/base 2025-12-04T09:21:04.1282839Z * [new branch] gh/fffrog/183/head -> origin/gh/fffrog/183/head 2025-12-04T09:21:04.1283462Z * [new branch] gh/fffrog/183/orig -> origin/gh/fffrog/183/orig 2025-12-04T09:21:04.1284083Z * [new branch] gh/fxdawnn/10/base -> origin/gh/fxdawnn/10/base 2025-12-04T09:21:04.1284967Z * [new branch] gh/fxdawnn/10/head -> origin/gh/fxdawnn/10/head 2025-12-04T09:21:04.1285988Z * [new branch] gh/fxdawnn/10/orig -> origin/gh/fxdawnn/10/orig 2025-12-04T09:21:04.1287813Z * [new branch] gh/fxdawnn/11/base -> origin/gh/fxdawnn/11/base 2025-12-04T09:21:04.1289018Z * [new branch] gh/fxdawnn/11/head -> origin/gh/fxdawnn/11/head 2025-12-04T09:21:04.1290276Z * [new branch] gh/fxdawnn/11/orig -> origin/gh/fxdawnn/11/orig 2025-12-04T09:21:04.1292167Z * [new branch] gh/fxdawnn/12/base -> origin/gh/fxdawnn/12/base 2025-12-04T09:21:04.1293433Z * [new branch] gh/fxdawnn/12/head -> origin/gh/fxdawnn/12/head 2025-12-04T09:21:04.1294646Z * [new branch] gh/fxdawnn/12/orig -> origin/gh/fxdawnn/12/orig 2025-12-04T09:21:04.1296328Z * [new branch] gh/fxdawnn/13/base -> origin/gh/fxdawnn/13/base 2025-12-04T09:21:04.1297519Z * [new branch] gh/fxdawnn/13/head -> origin/gh/fxdawnn/13/head 2025-12-04T09:21:04.1298708Z * [new branch] gh/fxdawnn/13/orig -> origin/gh/fxdawnn/13/orig 2025-12-04T09:21:04.1300430Z * [new branch] gh/fxdawnn/14/base -> origin/gh/fxdawnn/14/base 2025-12-04T09:21:04.1302391Z * [new branch] gh/fxdawnn/14/head -> origin/gh/fxdawnn/14/head 2025-12-04T09:21:04.1303658Z * [new branch] gh/fxdawnn/14/orig -> origin/gh/fxdawnn/14/orig 2025-12-04T09:21:04.1305434Z * [new branch] gh/fxdawnn/15/base -> origin/gh/fxdawnn/15/base 2025-12-04T09:21:04.1306712Z * [new branch] gh/fxdawnn/15/head -> origin/gh/fxdawnn/15/head 2025-12-04T09:21:04.1308076Z * [new branch] gh/fxdawnn/15/orig -> origin/gh/fxdawnn/15/orig 2025-12-04T09:21:04.1309914Z * [new branch] gh/fxdawnn/6/base -> origin/gh/fxdawnn/6/base 2025-12-04T09:21:04.1311046Z * [new branch] gh/fxdawnn/6/head -> origin/gh/fxdawnn/6/head 2025-12-04T09:21:04.1312301Z * [new branch] gh/fxdawnn/6/orig -> origin/gh/fxdawnn/6/orig 2025-12-04T09:21:04.1314026Z * [new branch] gh/fxdawnn/7/base -> origin/gh/fxdawnn/7/base 2025-12-04T09:21:04.1315329Z * [new branch] gh/fxdawnn/7/head -> origin/gh/fxdawnn/7/head 2025-12-04T09:21:04.1316456Z * [new branch] gh/fxdawnn/7/orig -> origin/gh/fxdawnn/7/orig 2025-12-04T09:21:04.1318196Z * [new branch] gh/fxdawnn/9/base -> origin/gh/fxdawnn/9/base 2025-12-04T09:21:04.1319414Z * [new branch] gh/fxdawnn/9/head -> origin/gh/fxdawnn/9/head 2025-12-04T09:21:04.1320637Z * [new branch] gh/fxdawnn/9/orig -> origin/gh/fxdawnn/9/orig 2025-12-04T09:21:04.1322560Z * [new branch] gh/galv/1/base -> origin/gh/galv/1/base 2025-12-04T09:21:04.1323683Z * [new branch] gh/galv/1/head -> origin/gh/galv/1/head 2025-12-04T09:21:04.1325446Z * [new branch] gh/galv/1/orig -> origin/gh/galv/1/orig 2025-12-04T09:21:04.1327079Z * [new branch] gh/galv/2/base -> origin/gh/galv/2/base 2025-12-04T09:21:04.1328219Z * [new branch] gh/galv/2/head -> origin/gh/galv/2/head 2025-12-04T09:21:04.1329692Z * [new branch] gh/galv/2/orig -> origin/gh/galv/2/orig 2025-12-04T09:21:04.1331174Z * [new branch] gh/galv/3/base -> origin/gh/galv/3/base 2025-12-04T09:21:04.1332210Z * [new branch] gh/galv/3/head -> origin/gh/galv/3/head 2025-12-04T09:21:04.1333630Z * [new branch] gh/galv/3/orig -> origin/gh/galv/3/orig 2025-12-04T09:21:04.1335585Z * [new branch] gh/guangyey/134/base -> origin/gh/guangyey/134/base 2025-12-04T09:21:04.1336774Z * [new branch] gh/guangyey/134/head -> origin/gh/guangyey/134/head 2025-12-04T09:21:04.1342734Z * [new branch] gh/guangyey/134/orig -> origin/gh/guangyey/134/orig 2025-12-04T09:21:04.1343426Z * [new branch] gh/guangyey/163/base -> origin/gh/guangyey/163/base 2025-12-04T09:21:04.1344087Z * [new branch] gh/guangyey/163/head -> origin/gh/guangyey/163/head 2025-12-04T09:21:04.1344832Z * [new branch] gh/guangyey/163/orig -> origin/gh/guangyey/163/orig 2025-12-04T09:21:04.1345584Z * [new branch] gh/guangyey/168/base -> origin/gh/guangyey/168/base 2025-12-04T09:21:04.1346242Z * [new branch] gh/guangyey/168/head -> origin/gh/guangyey/168/head 2025-12-04T09:21:04.1347148Z * [new branch] gh/guangyey/168/orig -> origin/gh/guangyey/168/orig 2025-12-04T09:21:04.1348855Z * [new branch] gh/guangyey/169/base -> origin/gh/guangyey/169/base 2025-12-04T09:21:04.1350052Z * [new branch] gh/guangyey/169/head -> origin/gh/guangyey/169/head 2025-12-04T09:21:04.1351532Z * [new branch] gh/guangyey/169/orig -> origin/gh/guangyey/169/orig 2025-12-04T09:21:04.1353003Z * [new branch] gh/guangyey/170/base -> origin/gh/guangyey/170/base 2025-12-04T09:21:04.1354184Z * [new branch] gh/guangyey/170/head -> origin/gh/guangyey/170/head 2025-12-04T09:21:04.1355731Z * [new branch] gh/guangyey/170/orig -> origin/gh/guangyey/170/orig 2025-12-04T09:21:04.1357400Z * [new branch] gh/guangyey/171/base -> origin/gh/guangyey/171/base 2025-12-04T09:21:04.1359095Z * [new branch] gh/guangyey/171/head -> origin/gh/guangyey/171/head 2025-12-04T09:21:04.1360036Z * [new branch] gh/guangyey/171/orig -> origin/gh/guangyey/171/orig 2025-12-04T09:21:04.1361709Z * [new branch] gh/guangyey/178/base -> origin/gh/guangyey/178/base 2025-12-04T09:21:04.1362909Z * [new branch] gh/guangyey/178/head -> origin/gh/guangyey/178/head 2025-12-04T09:21:04.1364054Z * [new branch] gh/guangyey/178/orig -> origin/gh/guangyey/178/orig 2025-12-04T09:21:04.1365629Z * [new branch] gh/guangyey/182/base -> origin/gh/guangyey/182/base 2025-12-04T09:21:04.1366801Z * [new branch] gh/guangyey/182/head -> origin/gh/guangyey/182/head 2025-12-04T09:21:04.1368021Z * [new branch] gh/guangyey/182/orig -> origin/gh/guangyey/182/orig 2025-12-04T09:21:04.1369608Z * [new branch] gh/guangyey/183/base -> origin/gh/guangyey/183/base 2025-12-04T09:21:04.1370761Z * [new branch] gh/guangyey/183/head -> origin/gh/guangyey/183/head 2025-12-04T09:21:04.1371970Z * [new branch] gh/guangyey/183/orig -> origin/gh/guangyey/183/orig 2025-12-04T09:21:04.1373628Z * [new branch] gh/guangyey/185/base -> origin/gh/guangyey/185/base 2025-12-04T09:21:04.1374882Z * [new branch] gh/guangyey/185/head -> origin/gh/guangyey/185/head 2025-12-04T09:21:04.1376093Z * [new branch] gh/guangyey/185/orig -> origin/gh/guangyey/185/orig 2025-12-04T09:21:04.1377750Z * [new branch] gh/guangyey/186/base -> origin/gh/guangyey/186/base 2025-12-04T09:21:04.1378902Z * [new branch] gh/guangyey/186/head -> origin/gh/guangyey/186/head 2025-12-04T09:21:04.1380112Z * [new branch] gh/guangyey/186/orig -> origin/gh/guangyey/186/orig 2025-12-04T09:21:04.1381704Z * [new branch] gh/guangyey/187/base -> origin/gh/guangyey/187/base 2025-12-04T09:21:04.1382836Z * [new branch] gh/guangyey/187/head -> origin/gh/guangyey/187/head 2025-12-04T09:21:04.1384040Z * [new branch] gh/guangyey/187/orig -> origin/gh/guangyey/187/orig 2025-12-04T09:21:04.1385661Z * [new branch] gh/guangyey/188/base -> origin/gh/guangyey/188/base 2025-12-04T09:21:04.1386873Z * [new branch] gh/guangyey/188/head -> origin/gh/guangyey/188/head 2025-12-04T09:21:04.1388493Z * [new branch] gh/guangyey/188/orig -> origin/gh/guangyey/188/orig 2025-12-04T09:21:04.1390142Z * [new branch] gh/guangyey/190/base -> origin/gh/guangyey/190/base 2025-12-04T09:21:04.1391429Z * [new branch] gh/guangyey/190/head -> origin/gh/guangyey/190/head 2025-12-04T09:21:04.1392671Z * [new branch] gh/guangyey/190/orig -> origin/gh/guangyey/190/orig 2025-12-04T09:21:04.1394405Z * [new branch] gh/guangyey/208/base -> origin/gh/guangyey/208/base 2025-12-04T09:21:04.1395560Z * [new branch] gh/guangyey/208/head -> origin/gh/guangyey/208/head 2025-12-04T09:21:04.1396783Z * [new branch] gh/guangyey/208/orig -> origin/gh/guangyey/208/orig 2025-12-04T09:21:04.1398485Z * [new branch] gh/guangyey/228/base -> origin/gh/guangyey/228/base 2025-12-04T09:21:04.1399790Z * [new branch] gh/guangyey/228/head -> origin/gh/guangyey/228/head 2025-12-04T09:21:04.1407540Z * [new branch] gh/guangyey/228/orig -> origin/gh/guangyey/228/orig 2025-12-04T09:21:04.1409094Z * [new branch] gh/guangyey/230/base -> origin/gh/guangyey/230/base 2025-12-04T09:21:04.1410287Z * [new branch] gh/guangyey/230/head -> origin/gh/guangyey/230/head 2025-12-04T09:21:04.1411537Z * [new branch] gh/guangyey/230/orig -> origin/gh/guangyey/230/orig 2025-12-04T09:21:04.1413397Z * [new branch] gh/guangyey/231/base -> origin/gh/guangyey/231/base 2025-12-04T09:21:04.1414700Z * [new branch] gh/guangyey/231/head -> origin/gh/guangyey/231/head 2025-12-04T09:21:04.1415900Z * [new branch] gh/guangyey/231/orig -> origin/gh/guangyey/231/orig 2025-12-04T09:21:04.1417544Z * [new branch] gh/guangyey/232/base -> origin/gh/guangyey/232/base 2025-12-04T09:21:04.1418707Z * [new branch] gh/guangyey/232/head -> origin/gh/guangyey/232/head 2025-12-04T09:21:04.1419881Z * [new branch] gh/guangyey/232/orig -> origin/gh/guangyey/232/orig 2025-12-04T09:21:04.1421540Z * [new branch] gh/guangyey/233/base -> origin/gh/guangyey/233/base 2025-12-04T09:21:04.1422697Z * [new branch] gh/guangyey/233/head -> origin/gh/guangyey/233/head 2025-12-04T09:21:04.1423898Z * [new branch] gh/guangyey/233/orig -> origin/gh/guangyey/233/orig 2025-12-04T09:21:04.1425550Z * [new branch] gh/guangyey/234/base -> origin/gh/guangyey/234/base 2025-12-04T09:21:04.1426789Z * [new branch] gh/guangyey/234/head -> origin/gh/guangyey/234/head 2025-12-04T09:21:04.1428297Z * [new branch] gh/guangyey/234/orig -> origin/gh/guangyey/234/orig 2025-12-04T09:21:04.1430028Z * [new branch] gh/guangyey/235/base -> origin/gh/guangyey/235/base 2025-12-04T09:21:04.1431307Z * [new branch] gh/guangyey/235/head -> origin/gh/guangyey/235/head 2025-12-04T09:21:04.1432477Z * [new branch] gh/guangyey/235/orig -> origin/gh/guangyey/235/orig 2025-12-04T09:21:04.1434521Z * [new branch] gh/guangyey/236/base -> origin/gh/guangyey/236/base 2025-12-04T09:21:04.1435872Z * [new branch] gh/guangyey/236/head -> origin/gh/guangyey/236/head 2025-12-04T09:21:04.1437041Z * [new branch] gh/guangyey/236/orig -> origin/gh/guangyey/236/orig 2025-12-04T09:21:04.1438782Z * [new branch] gh/guangyey/237/base -> origin/gh/guangyey/237/base 2025-12-04T09:21:04.1440023Z * [new branch] gh/guangyey/237/head -> origin/gh/guangyey/237/head 2025-12-04T09:21:04.1441235Z * [new branch] gh/guangyey/237/orig -> origin/gh/guangyey/237/orig 2025-12-04T09:21:04.1442907Z * [new branch] gh/guangyey/238/base -> origin/gh/guangyey/238/base 2025-12-04T09:21:04.1444030Z * [new branch] gh/guangyey/238/head -> origin/gh/guangyey/238/head 2025-12-04T09:21:04.1445669Z * [new branch] gh/guangyey/239/base -> origin/gh/guangyey/239/base 2025-12-04T09:21:04.1446814Z * [new branch] gh/guangyey/239/head -> origin/gh/guangyey/239/head 2025-12-04T09:21:04.1448458Z * [new branch] gh/guangyey/239/orig -> origin/gh/guangyey/239/orig 2025-12-04T09:21:04.1449767Z * [new branch] gh/guangyey/240/base -> origin/gh/guangyey/240/base 2025-12-04T09:21:04.1450986Z * [new branch] gh/guangyey/240/head -> origin/gh/guangyey/240/head 2025-12-04T09:21:04.1452170Z * [new branch] gh/guangyey/240/orig -> origin/gh/guangyey/240/orig 2025-12-04T09:21:04.1453865Z * [new branch] gh/guangyey/241/base -> origin/gh/guangyey/241/base 2025-12-04T09:21:04.1455384Z * [new branch] gh/guangyey/241/head -> origin/gh/guangyey/241/head 2025-12-04T09:21:04.1456620Z * [new branch] gh/guangyey/241/orig -> origin/gh/guangyey/241/orig 2025-12-04T09:21:04.1458340Z * [new branch] gh/guangyey/242/base -> origin/gh/guangyey/242/base 2025-12-04T09:21:04.1459428Z * [new branch] gh/guangyey/242/head -> origin/gh/guangyey/242/head 2025-12-04T09:21:04.1460638Z * [new branch] gh/guangyey/242/orig -> origin/gh/guangyey/242/orig 2025-12-04T09:21:04.1462384Z * [new branch] gh/guangyey/243/base -> origin/gh/guangyey/243/base 2025-12-04T09:21:04.1468567Z * [new branch] gh/guangyey/243/head -> origin/gh/guangyey/243/head 2025-12-04T09:21:04.1469869Z * [new branch] gh/guangyey/243/orig -> origin/gh/guangyey/243/orig 2025-12-04T09:21:04.1471731Z * [new branch] gh/guangyey/244/base -> origin/gh/guangyey/244/base 2025-12-04T09:21:04.1472938Z * [new branch] gh/guangyey/244/head -> origin/gh/guangyey/244/head 2025-12-04T09:21:04.1474162Z * [new branch] gh/guangyey/244/orig -> origin/gh/guangyey/244/orig 2025-12-04T09:21:04.1475914Z * [new branch] gh/guangyey/245/base -> origin/gh/guangyey/245/base 2025-12-04T09:21:04.1477106Z * [new branch] gh/guangyey/245/head -> origin/gh/guangyey/245/head 2025-12-04T09:21:04.1478335Z * [new branch] gh/guangyey/245/orig -> origin/gh/guangyey/245/orig 2025-12-04T09:21:04.1480156Z * [new branch] gh/guangyey/246/base -> origin/gh/guangyey/246/base 2025-12-04T09:21:04.1481302Z * [new branch] gh/guangyey/246/head -> origin/gh/guangyey/246/head 2025-12-04T09:21:04.1482523Z * [new branch] gh/guangyey/246/orig -> origin/gh/guangyey/246/orig 2025-12-04T09:21:04.1484203Z * [new branch] gh/guangyey/247/base -> origin/gh/guangyey/247/base 2025-12-04T09:21:04.1485350Z * [new branch] gh/guangyey/247/head -> origin/gh/guangyey/247/head 2025-12-04T09:21:04.1486632Z * [new branch] gh/guangyey/247/orig -> origin/gh/guangyey/247/orig 2025-12-04T09:21:04.1488311Z * [new branch] gh/guangyey/248/base -> origin/gh/guangyey/248/base 2025-12-04T09:21:04.1489543Z * [new branch] gh/guangyey/248/head -> origin/gh/guangyey/248/head 2025-12-04T09:21:04.1490631Z * [new branch] gh/guangyey/248/orig -> origin/gh/guangyey/248/orig 2025-12-04T09:21:04.1492293Z * [new branch] gh/guangyey/249/base -> origin/gh/guangyey/249/base 2025-12-04T09:21:04.1493545Z * [new branch] gh/guangyey/249/head -> origin/gh/guangyey/249/head 2025-12-04T09:21:04.1494799Z * [new branch] gh/guangyey/249/orig -> origin/gh/guangyey/249/orig 2025-12-04T09:21:04.1496488Z * [new branch] gh/guangyey/250/base -> origin/gh/guangyey/250/base 2025-12-04T09:21:04.1497663Z * [new branch] gh/guangyey/250/head -> origin/gh/guangyey/250/head 2025-12-04T09:21:04.1498908Z * [new branch] gh/guangyey/250/orig -> origin/gh/guangyey/250/orig 2025-12-04T09:21:04.1500504Z * [new branch] gh/guangyey/251/base -> origin/gh/guangyey/251/base 2025-12-04T09:21:04.1502652Z * [new branch] gh/guangyey/251/head -> origin/gh/guangyey/251/head 2025-12-04T09:21:04.1503908Z * [new branch] gh/guangyey/251/orig -> origin/gh/guangyey/251/orig 2025-12-04T09:21:04.1505642Z * [new branch] gh/guangyey/252/base -> origin/gh/guangyey/252/base 2025-12-04T09:21:04.1506983Z * [new branch] gh/guangyey/252/head -> origin/gh/guangyey/252/head 2025-12-04T09:21:04.1508342Z * [new branch] gh/guangyey/252/orig -> origin/gh/guangyey/252/orig 2025-12-04T09:21:04.1510091Z * [new branch] gh/guangyey/253/base -> origin/gh/guangyey/253/base 2025-12-04T09:21:04.1511293Z * [new branch] gh/guangyey/253/head -> origin/gh/guangyey/253/head 2025-12-04T09:21:04.1512568Z * [new branch] gh/guangyey/253/orig -> origin/gh/guangyey/253/orig 2025-12-04T09:21:04.1514252Z * [new branch] gh/guangyey/254/base -> origin/gh/guangyey/254/base 2025-12-04T09:21:04.1515438Z * [new branch] gh/guangyey/254/head -> origin/gh/guangyey/254/head 2025-12-04T09:21:04.1516660Z * [new branch] gh/guangyey/254/orig -> origin/gh/guangyey/254/orig 2025-12-04T09:21:04.1518492Z * [new branch] gh/guangyey/255/base -> origin/gh/guangyey/255/base 2025-12-04T09:21:04.1519655Z * [new branch] gh/guangyey/255/head -> origin/gh/guangyey/255/head 2025-12-04T09:21:04.1521151Z * [new branch] gh/guangyey/255/orig -> origin/gh/guangyey/255/orig 2025-12-04T09:21:04.1523113Z * [new branch] gh/guilhermeleobas/107/base -> origin/gh/guilhermeleobas/107/base 2025-12-04T09:21:04.1524270Z * [new branch] gh/guilhermeleobas/107/head -> origin/gh/guilhermeleobas/107/head 2025-12-04T09:21:04.1530272Z * [new branch] gh/guilhermeleobas/107/orig -> origin/gh/guilhermeleobas/107/orig 2025-12-04T09:21:04.1531038Z * [new branch] gh/guilhermeleobas/108/base -> origin/gh/guilhermeleobas/108/base 2025-12-04T09:21:04.1531894Z * [new branch] gh/guilhermeleobas/108/head -> origin/gh/guilhermeleobas/108/head 2025-12-04T09:21:04.1532648Z * [new branch] gh/guilhermeleobas/108/orig -> origin/gh/guilhermeleobas/108/orig 2025-12-04T09:21:04.1533405Z * [new branch] gh/guilhermeleobas/150/base -> origin/gh/guilhermeleobas/150/base 2025-12-04T09:21:04.1534479Z * [new branch] gh/guilhermeleobas/150/head -> origin/gh/guilhermeleobas/150/head 2025-12-04T09:21:04.1535746Z * [new branch] gh/guilhermeleobas/150/orig -> origin/gh/guilhermeleobas/150/orig 2025-12-04T09:21:04.1537633Z * [new branch] gh/guilhermeleobas/168/base -> origin/gh/guilhermeleobas/168/base 2025-12-04T09:21:04.1538876Z * [new branch] gh/guilhermeleobas/168/head -> origin/gh/guilhermeleobas/168/head 2025-12-04T09:21:04.1540064Z * [new branch] gh/guilhermeleobas/168/orig -> origin/gh/guilhermeleobas/168/orig 2025-12-04T09:21:04.1541782Z * [new branch] gh/guilhermeleobas/169/base -> origin/gh/guilhermeleobas/169/base 2025-12-04T09:21:04.1542840Z * [new branch] gh/guilhermeleobas/169/head -> origin/gh/guilhermeleobas/169/head 2025-12-04T09:21:04.1544021Z * [new branch] gh/guilhermeleobas/169/orig -> origin/gh/guilhermeleobas/169/orig 2025-12-04T09:21:04.1545994Z * [new branch] gh/guilhermeleobas/170/base -> origin/gh/guilhermeleobas/170/base 2025-12-04T09:21:04.1547603Z * [new branch] gh/guilhermeleobas/170/head -> origin/gh/guilhermeleobas/170/head 2025-12-04T09:21:04.1548812Z * [new branch] gh/guilhermeleobas/170/orig -> origin/gh/guilhermeleobas/170/orig 2025-12-04T09:21:04.1550534Z * [new branch] gh/guilhermeleobas/171/base -> origin/gh/guilhermeleobas/171/base 2025-12-04T09:21:04.1551711Z * [new branch] gh/guilhermeleobas/171/head -> origin/gh/guilhermeleobas/171/head 2025-12-04T09:21:04.1552924Z * [new branch] gh/guilhermeleobas/171/orig -> origin/gh/guilhermeleobas/171/orig 2025-12-04T09:21:04.1554573Z * [new branch] gh/guilhermeleobas/173/base -> origin/gh/guilhermeleobas/173/base 2025-12-04T09:21:04.1556001Z * [new branch] gh/guilhermeleobas/173/head -> origin/gh/guilhermeleobas/173/head 2025-12-04T09:21:04.1557228Z * [new branch] gh/guilhermeleobas/173/orig -> origin/gh/guilhermeleobas/173/orig 2025-12-04T09:21:04.1559052Z * [new branch] gh/guilhermeleobas/193/base -> origin/gh/guilhermeleobas/193/base 2025-12-04T09:21:04.1560196Z * [new branch] gh/guilhermeleobas/193/head -> origin/gh/guilhermeleobas/193/head 2025-12-04T09:21:04.1561554Z * [new branch] gh/guilhermeleobas/193/orig -> origin/gh/guilhermeleobas/193/orig 2025-12-04T09:21:04.1563160Z * [new branch] gh/guilhermeleobas/204/base -> origin/gh/guilhermeleobas/204/base 2025-12-04T09:21:04.1564337Z * [new branch] gh/guilhermeleobas/204/head -> origin/gh/guilhermeleobas/204/head 2025-12-04T09:21:04.1565473Z * [new branch] gh/guilhermeleobas/204/orig -> origin/gh/guilhermeleobas/204/orig 2025-12-04T09:21:04.1567549Z * [new branch] gh/guilhermeleobas/211/base -> origin/gh/guilhermeleobas/211/base 2025-12-04T09:21:04.1568715Z * [new branch] gh/guilhermeleobas/211/head -> origin/gh/guilhermeleobas/211/head 2025-12-04T09:21:04.1569900Z * [new branch] gh/guilhermeleobas/211/orig -> origin/gh/guilhermeleobas/211/orig 2025-12-04T09:21:04.1571531Z * [new branch] gh/guilhermeleobas/226/base -> origin/gh/guilhermeleobas/226/base 2025-12-04T09:21:04.1572738Z * [new branch] gh/guilhermeleobas/226/head -> origin/gh/guilhermeleobas/226/head 2025-12-04T09:21:04.1573942Z * [new branch] gh/guilhermeleobas/226/orig -> origin/gh/guilhermeleobas/226/orig 2025-12-04T09:21:04.1575578Z * [new branch] gh/guilhermeleobas/236/base -> origin/gh/guilhermeleobas/236/base 2025-12-04T09:21:04.1576733Z * [new branch] gh/guilhermeleobas/236/head -> origin/gh/guilhermeleobas/236/head 2025-12-04T09:21:04.1577953Z * [new branch] gh/guilhermeleobas/236/orig -> origin/gh/guilhermeleobas/236/orig 2025-12-04T09:21:04.1579534Z * [new branch] gh/guilhermeleobas/247/base -> origin/gh/guilhermeleobas/247/base 2025-12-04T09:21:04.1580677Z * [new branch] gh/guilhermeleobas/247/head -> origin/gh/guilhermeleobas/247/head 2025-12-04T09:21:04.1581921Z * [new branch] gh/guilhermeleobas/247/orig -> origin/gh/guilhermeleobas/247/orig 2025-12-04T09:21:04.1583546Z * [new branch] gh/guilhermeleobas/248/base -> origin/gh/guilhermeleobas/248/base 2025-12-04T09:21:04.1584683Z * [new branch] gh/guilhermeleobas/248/head -> origin/gh/guilhermeleobas/248/head 2025-12-04T09:21:04.1586401Z * [new branch] gh/guilhermeleobas/248/orig -> origin/gh/guilhermeleobas/248/orig 2025-12-04T09:21:04.1592921Z * [new branch] gh/guilhermeleobas/250/base -> origin/gh/guilhermeleobas/250/base 2025-12-04T09:21:04.1593774Z * [new branch] gh/guilhermeleobas/250/head -> origin/gh/guilhermeleobas/250/head 2025-12-04T09:21:04.1594566Z * [new branch] gh/guilhermeleobas/250/orig -> origin/gh/guilhermeleobas/250/orig 2025-12-04T09:21:04.1595363Z * [new branch] gh/guilhermeleobas/253/base -> origin/gh/guilhermeleobas/253/base 2025-12-04T09:21:04.1596218Z * [new branch] gh/guilhermeleobas/253/head -> origin/gh/guilhermeleobas/253/head 2025-12-04T09:21:04.1596978Z * [new branch] gh/guilhermeleobas/253/orig -> origin/gh/guilhermeleobas/253/orig 2025-12-04T09:21:04.1597976Z * [new branch] gh/guilhermeleobas/254/base -> origin/gh/guilhermeleobas/254/base 2025-12-04T09:21:04.1599518Z * [new branch] gh/guilhermeleobas/254/head -> origin/gh/guilhermeleobas/254/head 2025-12-04T09:21:04.1600505Z * [new branch] gh/guilhermeleobas/254/orig -> origin/gh/guilhermeleobas/254/orig 2025-12-04T09:21:04.1608408Z * [new branch] gh/guilhermeleobas/255/base -> origin/gh/guilhermeleobas/255/base 2025-12-04T09:21:04.1609386Z * [new branch] gh/guilhermeleobas/255/head -> origin/gh/guilhermeleobas/255/head 2025-12-04T09:21:04.1610662Z * [new branch] gh/guilhermeleobas/255/orig -> origin/gh/guilhermeleobas/255/orig 2025-12-04T09:21:04.1612494Z * [new branch] gh/guilhermeleobas/256/base -> origin/gh/guilhermeleobas/256/base 2025-12-04T09:21:04.1613956Z * [new branch] gh/guilhermeleobas/256/head -> origin/gh/guilhermeleobas/256/head 2025-12-04T09:21:04.1616233Z * [new branch] gh/guilhermeleobas/256/orig -> origin/gh/guilhermeleobas/256/orig 2025-12-04T09:21:04.1617043Z * [new branch] gh/guilhermeleobas/257/base -> origin/gh/guilhermeleobas/257/base 2025-12-04T09:21:04.1618026Z * [new branch] gh/guilhermeleobas/257/head -> origin/gh/guilhermeleobas/257/head 2025-12-04T09:21:04.1619420Z * [new branch] gh/guilhermeleobas/257/orig -> origin/gh/guilhermeleobas/257/orig 2025-12-04T09:21:04.1621090Z * [new branch] gh/guilhermeleobas/258/base -> origin/gh/guilhermeleobas/258/base 2025-12-04T09:21:04.1622226Z * [new branch] gh/guilhermeleobas/258/head -> origin/gh/guilhermeleobas/258/head 2025-12-04T09:21:04.1623491Z * [new branch] gh/guilhermeleobas/258/orig -> origin/gh/guilhermeleobas/258/orig 2025-12-04T09:21:04.1625256Z * [new branch] gh/guilhermeleobas/259/base -> origin/gh/guilhermeleobas/259/base 2025-12-04T09:21:04.1626673Z * [new branch] gh/guilhermeleobas/259/head -> origin/gh/guilhermeleobas/259/head 2025-12-04T09:21:04.1628061Z * [new branch] gh/guilhermeleobas/259/orig -> origin/gh/guilhermeleobas/259/orig 2025-12-04T09:21:04.1629753Z * [new branch] gh/guilhermeleobas/260/base -> origin/gh/guilhermeleobas/260/base 2025-12-04T09:21:04.1631078Z * [new branch] gh/guilhermeleobas/260/head -> origin/gh/guilhermeleobas/260/head 2025-12-04T09:21:04.1632342Z * [new branch] gh/guilhermeleobas/260/orig -> origin/gh/guilhermeleobas/260/orig 2025-12-04T09:21:04.1634118Z * [new branch] gh/guilhermeleobas/261/base -> origin/gh/guilhermeleobas/261/base 2025-12-04T09:21:04.1635783Z * [new branch] gh/guilhermeleobas/261/head -> origin/gh/guilhermeleobas/261/head 2025-12-04T09:21:04.1636985Z * [new branch] gh/guilhermeleobas/261/orig -> origin/gh/guilhermeleobas/261/orig 2025-12-04T09:21:04.1638748Z * [new branch] gh/guilhermeleobas/262/base -> origin/gh/guilhermeleobas/262/base 2025-12-04T09:21:04.1640104Z * [new branch] gh/guilhermeleobas/262/head -> origin/gh/guilhermeleobas/262/head 2025-12-04T09:21:04.1641280Z * [new branch] gh/guilhermeleobas/262/orig -> origin/gh/guilhermeleobas/262/orig 2025-12-04T09:21:04.1643090Z * [new branch] gh/guilhermeleobas/263/base -> origin/gh/guilhermeleobas/263/base 2025-12-04T09:21:04.1644183Z * [new branch] gh/guilhermeleobas/263/head -> origin/gh/guilhermeleobas/263/head 2025-12-04T09:21:04.1645372Z * [new branch] gh/guilhermeleobas/263/orig -> origin/gh/guilhermeleobas/263/orig 2025-12-04T09:21:04.1647076Z * [new branch] gh/guilhermeleobas/264/base -> origin/gh/guilhermeleobas/264/base 2025-12-04T09:21:04.1648314Z * [new branch] gh/guilhermeleobas/264/head -> origin/gh/guilhermeleobas/264/head 2025-12-04T09:21:04.1649503Z * [new branch] gh/guilhermeleobas/264/orig -> origin/gh/guilhermeleobas/264/orig 2025-12-04T09:21:04.1656581Z * [new branch] gh/guilhermeleobas/265/base -> origin/gh/guilhermeleobas/265/base 2025-12-04T09:21:04.1657743Z * [new branch] gh/guilhermeleobas/265/head -> origin/gh/guilhermeleobas/265/head 2025-12-04T09:21:04.1659067Z * [new branch] gh/guilhermeleobas/265/orig -> origin/gh/guilhermeleobas/265/orig 2025-12-04T09:21:04.1660823Z * [new branch] gh/guilhermeleobas/266/base -> origin/gh/guilhermeleobas/266/base 2025-12-04T09:21:04.1662623Z * [new branch] gh/guilhermeleobas/266/head -> origin/gh/guilhermeleobas/266/head 2025-12-04T09:21:04.1663654Z * [new branch] gh/guilhermeleobas/266/orig -> origin/gh/guilhermeleobas/266/orig 2025-12-04T09:21:04.1665394Z * [new branch] gh/guilhermeleobas/267/base -> origin/gh/guilhermeleobas/267/base 2025-12-04T09:21:04.1666624Z * [new branch] gh/guilhermeleobas/267/head -> origin/gh/guilhermeleobas/267/head 2025-12-04T09:21:04.1668144Z * [new branch] gh/guilhermeleobas/267/orig -> origin/gh/guilhermeleobas/267/orig 2025-12-04T09:21:04.1670367Z * [new branch] gh/hameerabbasi/1/base -> origin/gh/hameerabbasi/1/base 2025-12-04T09:21:04.1671627Z * [new branch] gh/hameerabbasi/1/head -> origin/gh/hameerabbasi/1/head 2025-12-04T09:21:04.1673193Z * [new branch] gh/hameerabbasi/2/base -> origin/gh/hameerabbasi/2/base 2025-12-04T09:21:04.1674421Z * [new branch] gh/hameerabbasi/2/head -> origin/gh/hameerabbasi/2/head 2025-12-04T09:21:04.1675682Z * [new branch] gh/hameerabbasi/2/orig -> origin/gh/hameerabbasi/2/orig 2025-12-04T09:21:04.1677292Z * [new branch] gh/hameerabbasi/3/base -> origin/gh/hameerabbasi/3/base 2025-12-04T09:21:04.1678577Z * [new branch] gh/hameerabbasi/3/head -> origin/gh/hameerabbasi/3/head 2025-12-04T09:21:04.1680044Z * [new branch] gh/hameerabbasi/3/orig -> origin/gh/hameerabbasi/3/orig 2025-12-04T09:21:04.1681984Z * [new branch] gh/hameerabbasi/4/base -> origin/gh/hameerabbasi/4/base 2025-12-04T09:21:04.1683384Z * [new branch] gh/hameerabbasi/4/head -> origin/gh/hameerabbasi/4/head 2025-12-04T09:21:04.1684642Z * [new branch] gh/hameerabbasi/4/orig -> origin/gh/hameerabbasi/4/orig 2025-12-04T09:21:04.1686494Z * [new branch] gh/huydhn/1/next -> origin/gh/huydhn/1/next 2025-12-04T09:21:04.1688017Z * [new branch] gh/huydhn/2/next -> origin/gh/huydhn/2/next 2025-12-04T09:21:04.1689595Z * [new branch] gh/huydhn/3/next -> origin/gh/huydhn/3/next 2025-12-04T09:21:04.1691334Z * [new branch] gh/huydhn/4/next -> origin/gh/huydhn/4/next 2025-12-04T09:21:04.1692875Z * [new branch] gh/huydhn/5/next -> origin/gh/huydhn/5/next 2025-12-04T09:21:04.1694755Z * [new branch] gh/huydhn/6/next -> origin/gh/huydhn/6/next 2025-12-04T09:21:04.1699133Z * [new branch] gh/int3/97/base -> origin/gh/int3/97/base 2025-12-04T09:21:04.1699716Z * [new branch] gh/int3/97/head -> origin/gh/int3/97/head 2025-12-04T09:21:04.1700490Z * [new branch] gh/isuruf/101/base -> origin/gh/isuruf/101/base 2025-12-04T09:21:04.1701778Z * [new branch] gh/isuruf/101/head -> origin/gh/isuruf/101/head 2025-12-04T09:21:04.1703714Z * [new branch] gh/isuruf/146/base -> origin/gh/isuruf/146/base 2025-12-04T09:21:04.1704948Z * [new branch] gh/isuruf/146/head -> origin/gh/isuruf/146/head 2025-12-04T09:21:04.1706208Z * [new branch] gh/isuruf/146/orig -> origin/gh/isuruf/146/orig 2025-12-04T09:21:04.1708085Z * [new branch] gh/isuruf/158/base -> origin/gh/isuruf/158/base 2025-12-04T09:21:04.1709235Z * [new branch] gh/isuruf/158/head -> origin/gh/isuruf/158/head 2025-12-04T09:21:04.1710947Z * [new branch] gh/isuruf/159/base -> origin/gh/isuruf/159/base 2025-12-04T09:21:04.1712034Z * [new branch] gh/isuruf/159/head -> origin/gh/isuruf/159/head 2025-12-04T09:21:04.1718878Z * [new branch] gh/isuruf/160/base -> origin/gh/isuruf/160/base 2025-12-04T09:21:04.1720014Z * [new branch] gh/isuruf/160/head -> origin/gh/isuruf/160/head 2025-12-04T09:21:04.1721459Z * [new branch] gh/isuruf/160/orig -> origin/gh/isuruf/160/orig 2025-12-04T09:21:04.1723114Z * [new branch] gh/isuruf/81/base -> origin/gh/isuruf/81/base 2025-12-04T09:21:04.1724251Z * [new branch] gh/isuruf/81/head -> origin/gh/isuruf/81/head 2025-12-04T09:21:04.1725570Z * [new branch] gh/isuruf/81/orig -> origin/gh/isuruf/81/orig 2025-12-04T09:21:04.1727540Z * [new branch] gh/jamesjwu/176/base -> origin/gh/jamesjwu/176/base 2025-12-04T09:21:04.1728699Z * [new branch] gh/jamesjwu/176/head -> origin/gh/jamesjwu/176/head 2025-12-04T09:21:04.1729876Z * [new branch] gh/jamesjwu/176/orig -> origin/gh/jamesjwu/176/orig 2025-12-04T09:21:04.1731504Z * [new branch] gh/jamesjwu/187/base -> origin/gh/jamesjwu/187/base 2025-12-04T09:21:04.1732643Z * [new branch] gh/jamesjwu/187/head -> origin/gh/jamesjwu/187/head 2025-12-04T09:21:04.1733806Z * [new branch] gh/jamesjwu/187/orig -> origin/gh/jamesjwu/187/orig 2025-12-04T09:21:04.1735493Z * [new branch] gh/jamesjwu/196/base -> origin/gh/jamesjwu/196/base 2025-12-04T09:21:04.1736896Z * [new branch] gh/jamesjwu/196/head -> origin/gh/jamesjwu/196/head 2025-12-04T09:21:04.1738049Z * [new branch] gh/jamesjwu/196/orig -> origin/gh/jamesjwu/196/orig 2025-12-04T09:21:04.1739651Z * [new branch] gh/jamesjwu/198/base -> origin/gh/jamesjwu/198/base 2025-12-04T09:21:04.1740799Z * [new branch] gh/jamesjwu/198/head -> origin/gh/jamesjwu/198/head 2025-12-04T09:21:04.1741983Z * [new branch] gh/jamesjwu/198/orig -> origin/gh/jamesjwu/198/orig 2025-12-04T09:21:04.1743712Z * [new branch] gh/jamesjwu/207/base -> origin/gh/jamesjwu/207/base 2025-12-04T09:21:04.1745136Z * [new branch] gh/jamesjwu/207/head -> origin/gh/jamesjwu/207/head 2025-12-04T09:21:04.1746254Z * [new branch] gh/jamesjwu/207/orig -> origin/gh/jamesjwu/207/orig 2025-12-04T09:21:04.1748416Z * [new branch] gh/jamesjwu/208/base -> origin/gh/jamesjwu/208/base 2025-12-04T09:21:04.1749634Z * [new branch] gh/jamesjwu/208/head -> origin/gh/jamesjwu/208/head 2025-12-04T09:21:04.1750840Z * [new branch] gh/jamesjwu/208/orig -> origin/gh/jamesjwu/208/orig 2025-12-04T09:21:04.1753018Z * [new branch] gh/jamesjwu/52/base -> origin/gh/jamesjwu/52/base 2025-12-04T09:21:04.1753869Z * [new branch] gh/jamesjwu/52/head -> origin/gh/jamesjwu/52/head 2025-12-04T09:21:04.1755489Z * [new branch] gh/jamesjwu/53/base -> origin/gh/jamesjwu/53/base 2025-12-04T09:21:04.1756532Z * [new branch] gh/jamesjwu/53/head -> origin/gh/jamesjwu/53/head 2025-12-04T09:21:04.1758051Z * [new branch] gh/jamesjwu/54/base -> origin/gh/jamesjwu/54/base 2025-12-04T09:21:04.1759302Z * [new branch] gh/jamesjwu/54/head -> origin/gh/jamesjwu/54/head 2025-12-04T09:21:04.1760795Z * [new branch] gh/jamesjwu/55/base -> origin/gh/jamesjwu/55/base 2025-12-04T09:21:04.1761894Z * [new branch] gh/jamesjwu/55/head -> origin/gh/jamesjwu/55/head 2025-12-04T09:21:04.1763395Z * [new branch] gh/jamesjwu/56/base -> origin/gh/jamesjwu/56/base 2025-12-04T09:21:04.1764502Z * [new branch] gh/jamesjwu/56/head -> origin/gh/jamesjwu/56/head 2025-12-04T09:21:04.1765939Z * [new branch] gh/jamesjwu/57/base -> origin/gh/jamesjwu/57/base 2025-12-04T09:21:04.1767013Z * [new branch] gh/jamesjwu/57/head -> origin/gh/jamesjwu/57/head 2025-12-04T09:21:04.1768654Z * [new branch] gh/jamesjwu/58/base -> origin/gh/jamesjwu/58/base 2025-12-04T09:21:04.1769718Z * [new branch] gh/jamesjwu/58/head -> origin/gh/jamesjwu/58/head 2025-12-04T09:21:04.1771164Z * [new branch] gh/jamesjwu/59/base -> origin/gh/jamesjwu/59/base 2025-12-04T09:21:04.1772291Z * [new branch] gh/jamesjwu/59/head -> origin/gh/jamesjwu/59/head 2025-12-04T09:21:04.1774118Z * [new branch] gh/jamesjwu/60/base -> origin/gh/jamesjwu/60/base 2025-12-04T09:21:04.1784852Z * [new branch] gh/jamesjwu/60/head -> origin/gh/jamesjwu/60/head 2025-12-04T09:21:04.1785493Z * [new branch] gh/jamesjwu/61/base -> origin/gh/jamesjwu/61/base 2025-12-04T09:21:04.1786198Z * [new branch] gh/jamesjwu/61/head -> origin/gh/jamesjwu/61/head 2025-12-04T09:21:04.1786938Z * [new branch] gh/jamesjwu/62/base -> origin/gh/jamesjwu/62/base 2025-12-04T09:21:04.1787647Z * [new branch] gh/jamesjwu/62/head -> origin/gh/jamesjwu/62/head 2025-12-04T09:21:04.1788284Z * [new branch] gh/jamesjwu/63/base -> origin/gh/jamesjwu/63/base 2025-12-04T09:21:04.1788953Z * [new branch] gh/jamesjwu/63/head -> origin/gh/jamesjwu/63/head 2025-12-04T09:21:04.1789653Z * [new branch] gh/jamesjwu/64/base -> origin/gh/jamesjwu/64/base 2025-12-04T09:21:04.1790358Z * [new branch] gh/jamesjwu/64/head -> origin/gh/jamesjwu/64/head 2025-12-04T09:21:04.1791004Z * [new branch] gh/jamesjwu/65/base -> origin/gh/jamesjwu/65/base 2025-12-04T09:21:04.1791646Z * [new branch] gh/jamesjwu/65/head -> origin/gh/jamesjwu/65/head 2025-12-04T09:21:04.1792505Z * [new branch] gh/janeyx99/165/base -> origin/gh/janeyx99/165/base 2025-12-04T09:21:04.1793787Z * [new branch] gh/janeyx99/165/head -> origin/gh/janeyx99/165/head 2025-12-04T09:21:04.1795000Z * [new branch] gh/janeyx99/165/orig -> origin/gh/janeyx99/165/orig 2025-12-04T09:21:04.1796541Z * [new branch] gh/janeyx99/201/base -> origin/gh/janeyx99/201/base 2025-12-04T09:21:04.1797744Z * [new branch] gh/janeyx99/201/head -> origin/gh/janeyx99/201/head 2025-12-04T09:21:04.1799146Z * [new branch] gh/janeyx99/201/orig -> origin/gh/janeyx99/201/orig 2025-12-04T09:21:04.1801797Z * [new branch] gh/janeyx99/225/base -> origin/gh/janeyx99/225/base 2025-12-04T09:21:04.1809944Z * [new branch] gh/janeyx99/225/head -> origin/gh/janeyx99/225/head 2025-12-04T09:21:04.1811209Z * [new branch] gh/janeyx99/225/orig -> origin/gh/janeyx99/225/orig 2025-12-04T09:21:04.1812982Z * [new branch] gh/janeyx99/299/base -> origin/gh/janeyx99/299/base 2025-12-04T09:21:04.1814362Z * [new branch] gh/janeyx99/299/head -> origin/gh/janeyx99/299/head 2025-12-04T09:21:04.1816021Z * [new branch] gh/janeyx99/299/orig -> origin/gh/janeyx99/299/orig 2025-12-04T09:21:04.1818050Z * [new branch] gh/janeyx99/302/base -> origin/gh/janeyx99/302/base 2025-12-04T09:21:04.1819255Z * [new branch] gh/janeyx99/302/head -> origin/gh/janeyx99/302/head 2025-12-04T09:21:04.1820776Z * [new branch] gh/janeyx99/303/base -> origin/gh/janeyx99/303/base 2025-12-04T09:21:04.1821829Z * [new branch] gh/janeyx99/303/head -> origin/gh/janeyx99/303/head 2025-12-04T09:21:04.1823613Z * [new branch] gh/janeyx99/305/base -> origin/gh/janeyx99/305/base 2025-12-04T09:21:04.1824805Z * [new branch] gh/janeyx99/305/head -> origin/gh/janeyx99/305/head 2025-12-04T09:21:04.1826288Z * [new branch] gh/janeyx99/306/base -> origin/gh/janeyx99/306/base 2025-12-04T09:21:04.1827679Z * [new branch] gh/janeyx99/306/head -> origin/gh/janeyx99/306/head 2025-12-04T09:21:04.1829400Z * [new branch] gh/janeyx99/314/base -> origin/gh/janeyx99/314/base 2025-12-04T09:21:04.1830651Z * [new branch] gh/janeyx99/314/head -> origin/gh/janeyx99/314/head 2025-12-04T09:21:04.1831885Z * [new branch] gh/janeyx99/314/orig -> origin/gh/janeyx99/314/orig 2025-12-04T09:21:04.1833575Z * [new branch] gh/janeyx99/315/base -> origin/gh/janeyx99/315/base 2025-12-04T09:21:04.1834787Z * [new branch] gh/janeyx99/315/head -> origin/gh/janeyx99/315/head 2025-12-04T09:21:04.1836026Z * [new branch] gh/janeyx99/315/orig -> origin/gh/janeyx99/315/orig 2025-12-04T09:21:04.1842547Z * [new branch] gh/janeyx99/316/base -> origin/gh/janeyx99/316/base 2025-12-04T09:21:04.1843195Z * [new branch] gh/janeyx99/316/head -> origin/gh/janeyx99/316/head 2025-12-04T09:21:04.1843948Z * [new branch] gh/janeyx99/316/orig -> origin/gh/janeyx99/316/orig 2025-12-04T09:21:04.1844637Z * [new branch] gh/janeyx99/317/base -> origin/gh/janeyx99/317/base 2025-12-04T09:21:04.1845304Z * [new branch] gh/janeyx99/317/head -> origin/gh/janeyx99/317/head 2025-12-04T09:21:04.1845945Z * [new branch] gh/janeyx99/317/orig -> origin/gh/janeyx99/317/orig 2025-12-04T09:21:04.1846679Z * [new branch] gh/janeyx99/325/base -> origin/gh/janeyx99/325/base 2025-12-04T09:21:04.1847845Z * [new branch] gh/janeyx99/325/head -> origin/gh/janeyx99/325/head 2025-12-04T09:21:04.1849121Z * [new branch] gh/janeyx99/325/orig -> origin/gh/janeyx99/325/orig 2025-12-04T09:21:04.1850870Z * [new branch] gh/janeyx99/327/base -> origin/gh/janeyx99/327/base 2025-12-04T09:21:04.1852075Z * [new branch] gh/janeyx99/327/head -> origin/gh/janeyx99/327/head 2025-12-04T09:21:04.1853231Z * [new branch] gh/janeyx99/327/orig -> origin/gh/janeyx99/327/orig 2025-12-04T09:21:04.1854897Z * [new branch] gh/janeyx99/328/base -> origin/gh/janeyx99/328/base 2025-12-04T09:21:04.1856126Z * [new branch] gh/janeyx99/328/head -> origin/gh/janeyx99/328/head 2025-12-04T09:21:04.1857495Z * [new branch] gh/janeyx99/328/orig -> origin/gh/janeyx99/328/orig 2025-12-04T09:21:04.1858943Z * [new branch] gh/janeyx99/329/base -> origin/gh/janeyx99/329/base 2025-12-04T09:21:04.1860129Z * [new branch] gh/janeyx99/329/head -> origin/gh/janeyx99/329/head 2025-12-04T09:21:04.1861333Z * [new branch] gh/janeyx99/329/orig -> origin/gh/janeyx99/329/orig 2025-12-04T09:21:04.1863517Z * [new branch] gh/janeyx99/330/base -> origin/gh/janeyx99/330/base 2025-12-04T09:21:04.1864682Z * [new branch] gh/janeyx99/330/head -> origin/gh/janeyx99/330/head 2025-12-04T09:21:04.1865808Z * [new branch] gh/janeyx99/330/orig -> origin/gh/janeyx99/330/orig 2025-12-04T09:21:04.1867883Z * [new branch] gh/janeyx99/331/base -> origin/gh/janeyx99/331/base 2025-12-04T09:21:04.1869089Z * [new branch] gh/janeyx99/331/head -> origin/gh/janeyx99/331/head 2025-12-04T09:21:04.1870323Z * [new branch] gh/janeyx99/331/orig -> origin/gh/janeyx99/331/orig 2025-12-04T09:21:04.1872048Z * [new branch] gh/janeyx99/332/base -> origin/gh/janeyx99/332/base 2025-12-04T09:21:04.1873295Z * [new branch] gh/janeyx99/332/head -> origin/gh/janeyx99/332/head 2025-12-04T09:21:04.1874626Z * [new branch] gh/janeyx99/332/orig -> origin/gh/janeyx99/332/orig 2025-12-04T09:21:04.1876212Z * [new branch] gh/janeyx99/333/base -> origin/gh/janeyx99/333/base 2025-12-04T09:21:04.1877375Z * [new branch] gh/janeyx99/333/head -> origin/gh/janeyx99/333/head 2025-12-04T09:21:04.1878594Z * [new branch] gh/janeyx99/333/orig -> origin/gh/janeyx99/333/orig 2025-12-04T09:21:04.1880485Z * [new branch] gh/janeyx99/88/base -> origin/gh/janeyx99/88/base 2025-12-04T09:21:04.1881636Z * [new branch] gh/janeyx99/88/head -> origin/gh/janeyx99/88/head 2025-12-04T09:21:04.1882833Z * [new branch] gh/janeyx99/88/orig -> origin/gh/janeyx99/88/orig 2025-12-04T09:21:04.1884813Z * [new branch] gh/jansel/360/base -> origin/gh/jansel/360/base 2025-12-04T09:21:04.1886061Z * [new branch] gh/jansel/360/head -> origin/gh/jansel/360/head 2025-12-04T09:21:04.1887625Z * [new branch] gh/jansel/451/base -> origin/gh/jansel/451/base 2025-12-04T09:21:04.1889123Z * [new branch] gh/jansel/451/head -> origin/gh/jansel/451/head 2025-12-04T09:21:04.1890377Z * [new branch] gh/jansel/451/orig -> origin/gh/jansel/451/orig 2025-12-04T09:21:04.1892091Z * [new branch] gh/jansel/462/base -> origin/gh/jansel/462/base 2025-12-04T09:21:04.1893251Z * [new branch] gh/jansel/462/head -> origin/gh/jansel/462/head 2025-12-04T09:21:04.1894444Z * [new branch] gh/jansel/462/orig -> origin/gh/jansel/462/orig 2025-12-04T09:21:04.1896045Z * [new branch] gh/jansel/533/base -> origin/gh/jansel/533/base 2025-12-04T09:21:04.1897194Z * [new branch] gh/jansel/533/head -> origin/gh/jansel/533/head 2025-12-04T09:21:04.1898351Z * [new branch] gh/jansel/533/orig -> origin/gh/jansel/533/orig 2025-12-04T09:21:04.1905995Z * [new branch] gh/jansel/552/base -> origin/gh/jansel/552/base 2025-12-04T09:21:04.1907113Z * [new branch] gh/jansel/552/head -> origin/gh/jansel/552/head 2025-12-04T09:21:04.1908396Z * [new branch] gh/jansel/552/orig -> origin/gh/jansel/552/orig 2025-12-04T09:21:04.1910159Z * [new branch] gh/jansel/553/base -> origin/gh/jansel/553/base 2025-12-04T09:21:04.1911327Z * [new branch] gh/jansel/553/head -> origin/gh/jansel/553/head 2025-12-04T09:21:04.1912867Z * [new branch] gh/jansel/553/orig -> origin/gh/jansel/553/orig 2025-12-04T09:21:04.1914325Z * [new branch] gh/jansel/554/base -> origin/gh/jansel/554/base 2025-12-04T09:21:04.1915519Z * [new branch] gh/jansel/554/head -> origin/gh/jansel/554/head 2025-12-04T09:21:04.1916700Z * [new branch] gh/jansel/554/orig -> origin/gh/jansel/554/orig 2025-12-04T09:21:04.1918521Z * [new branch] gh/jansel/555/base -> origin/gh/jansel/555/base 2025-12-04T09:21:04.1919770Z * [new branch] gh/jansel/555/head -> origin/gh/jansel/555/head 2025-12-04T09:21:04.1921367Z * [new branch] gh/jansel/555/orig -> origin/gh/jansel/555/orig 2025-12-04T09:21:04.1922954Z * [new branch] gh/jansel/556/base -> origin/gh/jansel/556/base 2025-12-04T09:21:04.1924092Z * [new branch] gh/jansel/556/head -> origin/gh/jansel/556/head 2025-12-04T09:21:04.1925265Z * [new branch] gh/jansel/556/orig -> origin/gh/jansel/556/orig 2025-12-04T09:21:04.1926878Z * [new branch] gh/jansel/557/base -> origin/gh/jansel/557/base 2025-12-04T09:21:04.1928015Z * [new branch] gh/jansel/557/head -> origin/gh/jansel/557/head 2025-12-04T09:21:04.1929303Z * [new branch] gh/jansel/557/orig -> origin/gh/jansel/557/orig 2025-12-04T09:21:04.1930920Z * [new branch] gh/jansel/558/base -> origin/gh/jansel/558/base 2025-12-04T09:21:04.1932064Z * [new branch] gh/jansel/558/head -> origin/gh/jansel/558/head 2025-12-04T09:21:04.1933276Z * [new branch] gh/jansel/558/orig -> origin/gh/jansel/558/orig 2025-12-04T09:21:04.1934920Z * [new branch] gh/jansel/559/base -> origin/gh/jansel/559/base 2025-12-04T09:21:04.1936084Z * [new branch] gh/jansel/559/head -> origin/gh/jansel/559/head 2025-12-04T09:21:04.1937253Z * [new branch] gh/jansel/559/orig -> origin/gh/jansel/559/orig 2025-12-04T09:21:04.1938913Z * [new branch] gh/jansel/560/base -> origin/gh/jansel/560/base 2025-12-04T09:21:04.1940040Z * [new branch] gh/jansel/560/head -> origin/gh/jansel/560/head 2025-12-04T09:21:04.1941238Z * [new branch] gh/jansel/560/orig -> origin/gh/jansel/560/orig 2025-12-04T09:21:04.1942786Z * [new branch] gh/jansel/561/base -> origin/gh/jansel/561/base 2025-12-04T09:21:04.1943968Z * [new branch] gh/jansel/561/head -> origin/gh/jansel/561/head 2025-12-04T09:21:04.1945241Z * [new branch] gh/jansel/561/orig -> origin/gh/jansel/561/orig 2025-12-04T09:21:04.1947706Z * [new branch] gh/jansel/562/base -> origin/gh/jansel/562/base 2025-12-04T09:21:04.1948980Z * [new branch] gh/jansel/562/head -> origin/gh/jansel/562/head 2025-12-04T09:21:04.1950211Z * [new branch] gh/jansel/562/orig -> origin/gh/jansel/562/orig 2025-12-04T09:21:04.1952037Z * [new branch] gh/jansel/563/base -> origin/gh/jansel/563/base 2025-12-04T09:21:04.1952985Z * [new branch] gh/jansel/563/head -> origin/gh/jansel/563/head 2025-12-04T09:21:04.1954255Z * [new branch] gh/jansel/563/orig -> origin/gh/jansel/563/orig 2025-12-04T09:21:04.1956451Z * [new branch] gh/jansel/564/base -> origin/gh/jansel/564/base 2025-12-04T09:21:04.1957689Z * [new branch] gh/jansel/564/head -> origin/gh/jansel/564/head 2025-12-04T09:21:04.1958920Z * [new branch] gh/jansel/564/orig -> origin/gh/jansel/564/orig 2025-12-04T09:21:04.1960710Z * [new branch] gh/jansel/565/base -> origin/gh/jansel/565/base 2025-12-04T09:21:04.1961911Z * [new branch] gh/jansel/565/head -> origin/gh/jansel/565/head 2025-12-04T09:21:04.1968743Z * [new branch] gh/jansel/565/orig -> origin/gh/jansel/565/orig 2025-12-04T09:21:04.1970200Z * [new branch] gh/jansel/566/base -> origin/gh/jansel/566/base 2025-12-04T09:21:04.1971439Z * [new branch] gh/jansel/566/head -> origin/gh/jansel/566/head 2025-12-04T09:21:04.1972629Z * [new branch] gh/jansel/566/orig -> origin/gh/jansel/566/orig 2025-12-04T09:21:04.1974330Z * [new branch] gh/jansel/567/base -> origin/gh/jansel/567/base 2025-12-04T09:21:04.1975598Z * [new branch] gh/jansel/567/head -> origin/gh/jansel/567/head 2025-12-04T09:21:04.1976795Z * [new branch] gh/jansel/567/orig -> origin/gh/jansel/567/orig 2025-12-04T09:21:04.1978505Z * [new branch] gh/jansel/568/base -> origin/gh/jansel/568/base 2025-12-04T09:21:04.1979893Z * [new branch] gh/jansel/568/head -> origin/gh/jansel/568/head 2025-12-04T09:21:04.1981140Z * [new branch] gh/jansel/568/orig -> origin/gh/jansel/568/orig 2025-12-04T09:21:04.1982726Z * [new branch] gh/jansel/569/base -> origin/gh/jansel/569/base 2025-12-04T09:21:04.1983959Z * [new branch] gh/jansel/569/head -> origin/gh/jansel/569/head 2025-12-04T09:21:04.1985226Z * [new branch] gh/jansel/569/orig -> origin/gh/jansel/569/orig 2025-12-04T09:21:04.1987177Z * [new branch] gh/jansel/570/base -> origin/gh/jansel/570/base 2025-12-04T09:21:04.1988509Z * [new branch] gh/jansel/570/head -> origin/gh/jansel/570/head 2025-12-04T09:21:04.1989724Z * [new branch] gh/jansel/570/orig -> origin/gh/jansel/570/orig 2025-12-04T09:21:04.1991406Z * [new branch] gh/jansel/571/base -> origin/gh/jansel/571/base 2025-12-04T09:21:04.1992631Z * [new branch] gh/jansel/571/head -> origin/gh/jansel/571/head 2025-12-04T09:21:04.1993867Z * [new branch] gh/jansel/571/orig -> origin/gh/jansel/571/orig 2025-12-04T09:21:04.1995441Z * [new branch] gh/jansel/572/base -> origin/gh/jansel/572/base 2025-12-04T09:21:04.1996695Z * [new branch] gh/jansel/572/head -> origin/gh/jansel/572/head 2025-12-04T09:21:04.1997919Z * [new branch] gh/jansel/572/orig -> origin/gh/jansel/572/orig 2025-12-04T09:21:04.1999772Z * [new branch] gh/jansel/573/base -> origin/gh/jansel/573/base 2025-12-04T09:21:04.2001510Z * [new branch] gh/jansel/573/head -> origin/gh/jansel/573/head 2025-12-04T09:21:04.2003131Z * [new branch] gh/jansel/573/orig -> origin/gh/jansel/573/orig 2025-12-04T09:21:04.2005167Z * [new branch] gh/jansel/574/base -> origin/gh/jansel/574/base 2025-12-04T09:21:04.2006391Z * [new branch] gh/jansel/574/head -> origin/gh/jansel/574/head 2025-12-04T09:21:04.2007653Z * [new branch] gh/jansel/574/orig -> origin/gh/jansel/574/orig 2025-12-04T09:21:04.2009372Z * [new branch] gh/jansel/575/base -> origin/gh/jansel/575/base 2025-12-04T09:21:04.2010637Z * [new branch] gh/jansel/575/head -> origin/gh/jansel/575/head 2025-12-04T09:21:04.2012068Z * [new branch] gh/jansel/575/orig -> origin/gh/jansel/575/orig 2025-12-04T09:21:04.2013542Z * [new branch] gh/jansel/576/base -> origin/gh/jansel/576/base 2025-12-04T09:21:04.2014873Z * [new branch] gh/jansel/576/head -> origin/gh/jansel/576/head 2025-12-04T09:21:04.2016025Z * [new branch] gh/jansel/576/orig -> origin/gh/jansel/576/orig 2025-12-04T09:21:04.2018051Z * [new branch] gh/jbschlosser/247/base -> origin/gh/jbschlosser/247/base 2025-12-04T09:21:04.2019354Z * [new branch] gh/jbschlosser/247/head -> origin/gh/jbschlosser/247/head 2025-12-04T09:21:04.2020571Z * [new branch] gh/jbschlosser/247/orig -> origin/gh/jbschlosser/247/orig 2025-12-04T09:21:04.2022215Z * [new branch] gh/jbschlosser/250/base -> origin/gh/jbschlosser/250/base 2025-12-04T09:21:04.2023328Z * [new branch] gh/jbschlosser/250/head -> origin/gh/jbschlosser/250/head 2025-12-04T09:21:04.2024524Z * [new branch] gh/jbschlosser/250/orig -> origin/gh/jbschlosser/250/orig 2025-12-04T09:21:04.2030338Z * [new branch] gh/jerryzh168/1/base -> origin/gh/jerryzh168/1/base 2025-12-04T09:21:04.2030611Z * [new branch] gh/jerryzh168/1/head -> origin/gh/jerryzh168/1/head 2025-12-04T09:21:04.2030890Z * [new branch] gh/jerryzh168/1/orig -> origin/gh/jerryzh168/1/orig 2025-12-04T09:21:04.2031748Z * [new branch] gh/jiayisunx/59/base -> origin/gh/jiayisunx/59/base 2025-12-04T09:21:04.2033036Z * [new branch] gh/jiayisunx/59/head -> origin/gh/jiayisunx/59/head 2025-12-04T09:21:04.2034246Z * [new branch] gh/jiayisunx/59/orig -> origin/gh/jiayisunx/59/orig 2025-12-04T09:21:04.2035900Z * [new branch] gh/jiayisunx/61/base -> origin/gh/jiayisunx/61/base 2025-12-04T09:21:04.2037111Z * [new branch] gh/jiayisunx/61/head -> origin/gh/jiayisunx/61/head 2025-12-04T09:21:04.2038454Z * [new branch] gh/jiayisunx/61/orig -> origin/gh/jiayisunx/61/orig 2025-12-04T09:21:04.2040082Z * [new branch] gh/jiayisunx/68/base -> origin/gh/jiayisunx/68/base 2025-12-04T09:21:04.2041213Z * [new branch] gh/jiayisunx/68/head -> origin/gh/jiayisunx/68/head 2025-12-04T09:21:04.2042419Z * [new branch] gh/jiayisunx/68/orig -> origin/gh/jiayisunx/68/orig 2025-12-04T09:21:04.2044047Z * [new branch] gh/jiayisunx/77/base -> origin/gh/jiayisunx/77/base 2025-12-04T09:21:04.2045204Z * [new branch] gh/jiayisunx/77/head -> origin/gh/jiayisunx/77/head 2025-12-04T09:21:04.2046673Z * [new branch] gh/jiayisunx/77/orig -> origin/gh/jiayisunx/77/orig 2025-12-04T09:21:04.2047989Z * [new branch] gh/jiayisunx/78/base -> origin/gh/jiayisunx/78/base 2025-12-04T09:21:04.2049174Z * [new branch] gh/jiayisunx/78/head -> origin/gh/jiayisunx/78/head 2025-12-04T09:21:04.2050343Z * [new branch] gh/jiayisunx/78/orig -> origin/gh/jiayisunx/78/orig 2025-12-04T09:21:04.2052073Z * [new branch] gh/jiayisunx/79/base -> origin/gh/jiayisunx/79/base 2025-12-04T09:21:04.2053711Z * [new branch] gh/jiayisunx/79/head -> origin/gh/jiayisunx/79/head 2025-12-04T09:21:04.2054916Z * [new branch] gh/jiayisunx/79/orig -> origin/gh/jiayisunx/79/orig 2025-12-04T09:21:04.2056547Z * [new branch] gh/jiayisunx/82/base -> origin/gh/jiayisunx/82/base 2025-12-04T09:21:04.2057739Z * [new branch] gh/jiayisunx/82/head -> origin/gh/jiayisunx/82/head 2025-12-04T09:21:04.2058962Z * [new branch] gh/jiayisunx/82/orig -> origin/gh/jiayisunx/82/orig 2025-12-04T09:21:04.2060514Z * [new branch] gh/jiayisunx/83/base -> origin/gh/jiayisunx/83/base 2025-12-04T09:21:04.2061719Z * [new branch] gh/jiayisunx/83/head -> origin/gh/jiayisunx/83/head 2025-12-04T09:21:04.2062885Z * [new branch] gh/jiayisunx/83/orig -> origin/gh/jiayisunx/83/orig 2025-12-04T09:21:04.2064359Z * [new branch] gh/jiayisunx/84/base -> origin/gh/jiayisunx/84/base 2025-12-04T09:21:04.2065567Z * [new branch] gh/jiayisunx/84/head -> origin/gh/jiayisunx/84/head 2025-12-04T09:21:04.2066864Z * [new branch] gh/jiayisunx/84/orig -> origin/gh/jiayisunx/84/orig 2025-12-04T09:21:04.2068900Z * [new branch] gh/jiayisunx/85/base -> origin/gh/jiayisunx/85/base 2025-12-04T09:21:04.2070089Z * [new branch] gh/jiayisunx/85/head -> origin/gh/jiayisunx/85/head 2025-12-04T09:21:04.2071321Z * [new branch] gh/jiayisunx/85/orig -> origin/gh/jiayisunx/85/orig 2025-12-04T09:21:04.2072900Z * [new branch] gh/jiayisunx/86/base -> origin/gh/jiayisunx/86/base 2025-12-04T09:21:04.2074124Z * [new branch] gh/jiayisunx/86/head -> origin/gh/jiayisunx/86/head 2025-12-04T09:21:04.2075513Z * [new branch] gh/jiayisunx/86/orig -> origin/gh/jiayisunx/86/orig 2025-12-04T09:21:04.2077041Z * [new branch] gh/jiayisunx/87/base -> origin/gh/jiayisunx/87/base 2025-12-04T09:21:04.2078234Z * [new branch] gh/jiayisunx/87/head -> origin/gh/jiayisunx/87/head 2025-12-04T09:21:04.2079545Z * [new branch] gh/jiayisunx/87/orig -> origin/gh/jiayisunx/87/orig 2025-12-04T09:21:04.2081125Z * [new branch] gh/jiayisunx/88/base -> origin/gh/jiayisunx/88/base 2025-12-04T09:21:04.2082803Z * [new branch] gh/jiayisunx/88/head -> origin/gh/jiayisunx/88/head 2025-12-04T09:21:04.2084064Z * [new branch] gh/jiayisunx/88/orig -> origin/gh/jiayisunx/88/orig 2025-12-04T09:21:04.2085791Z * [new branch] gh/jiayisunx/89/base -> origin/gh/jiayisunx/89/base 2025-12-04T09:21:04.2086935Z * [new branch] gh/jiayisunx/89/head -> origin/gh/jiayisunx/89/head 2025-12-04T09:21:04.2092863Z * [new branch] gh/jiayisunx/89/orig -> origin/gh/jiayisunx/89/orig 2025-12-04T09:21:04.2093144Z * [new branch] gh/jiayisunx/90/base -> origin/gh/jiayisunx/90/base 2025-12-04T09:21:04.2093423Z * [new branch] gh/jiayisunx/90/head -> origin/gh/jiayisunx/90/head 2025-12-04T09:21:04.2093684Z * [new branch] gh/jiayisunx/90/orig -> origin/gh/jiayisunx/90/orig 2025-12-04T09:21:04.2094547Z * [new branch] gh/jjwu@meta.com/1/base -> origin/gh/jjwu@meta.com/1/base 2025-12-04T09:21:04.2095802Z * [new branch] gh/jjwu@meta.com/1/head -> origin/gh/jjwu@meta.com/1/head 2025-12-04T09:21:04.2097737Z * [new branch] gh/jturney/1/base -> origin/gh/jturney/1/base 2025-12-04T09:21:04.2098951Z * [new branch] gh/jturney/1/head -> origin/gh/jturney/1/head 2025-12-04T09:21:04.2100291Z * [new branch] gh/jturney/1/orig -> origin/gh/jturney/1/orig 2025-12-04T09:21:04.2102572Z * [new branch] gh/jturney/2/base -> origin/gh/jturney/2/base 2025-12-04T09:21:04.2103770Z * [new branch] gh/jturney/2/head -> origin/gh/jturney/2/head 2025-12-04T09:21:04.2105012Z * [new branch] gh/jturney/2/orig -> origin/gh/jturney/2/orig 2025-12-04T09:21:04.2107428Z * [new branch] gh/karthickai/10/base -> origin/gh/karthickai/10/base 2025-12-04T09:21:04.2108748Z * [new branch] gh/karthickai/10/head -> origin/gh/karthickai/10/head 2025-12-04T09:21:04.2110270Z * [new branch] gh/karthickai/10/orig -> origin/gh/karthickai/10/orig 2025-12-04T09:21:04.2111985Z * [new branch] gh/karthickai/11/base -> origin/gh/karthickai/11/base 2025-12-04T09:21:04.2113291Z * [new branch] gh/karthickai/11/head -> origin/gh/karthickai/11/head 2025-12-04T09:21:04.2114537Z * [new branch] gh/karthickai/11/orig -> origin/gh/karthickai/11/orig 2025-12-04T09:21:04.2116597Z * [new branch] gh/karthickai/12/base -> origin/gh/karthickai/12/base 2025-12-04T09:21:04.2117994Z * [new branch] gh/karthickai/12/head -> origin/gh/karthickai/12/head 2025-12-04T09:21:04.2119397Z * [new branch] gh/karthickai/12/orig -> origin/gh/karthickai/12/orig 2025-12-04T09:21:04.2121006Z * [new branch] gh/karthickai/13/base -> origin/gh/karthickai/13/base 2025-12-04T09:21:04.2122253Z * [new branch] gh/karthickai/13/head -> origin/gh/karthickai/13/head 2025-12-04T09:21:04.2123478Z * [new branch] gh/karthickai/13/orig -> origin/gh/karthickai/13/orig 2025-12-04T09:21:04.2125238Z * [new branch] gh/karthickai/14/base -> origin/gh/karthickai/14/base 2025-12-04T09:21:04.2126535Z * [new branch] gh/karthickai/14/head -> origin/gh/karthickai/14/head 2025-12-04T09:21:04.2128290Z * [new branch] gh/karthickai/14/orig -> origin/gh/karthickai/14/orig 2025-12-04T09:21:04.2130527Z * [new branch] gh/karthickai/15/base -> origin/gh/karthickai/15/base 2025-12-04T09:21:04.2131814Z * [new branch] gh/karthickai/15/head -> origin/gh/karthickai/15/head 2025-12-04T09:21:04.2133040Z * [new branch] gh/karthickai/15/orig -> origin/gh/karthickai/15/orig 2025-12-04T09:21:04.2134556Z * [new branch] gh/karthickai/16/base -> origin/gh/karthickai/16/base 2025-12-04T09:21:04.2135873Z * [new branch] gh/karthickai/16/head -> origin/gh/karthickai/16/head 2025-12-04T09:21:04.2137075Z * [new branch] gh/karthickai/16/orig -> origin/gh/karthickai/16/orig 2025-12-04T09:21:04.2138604Z * [new branch] gh/karthickai/17/base -> origin/gh/karthickai/17/base 2025-12-04T09:21:04.2139703Z * [new branch] gh/karthickai/17/head -> origin/gh/karthickai/17/head 2025-12-04T09:21:04.2141373Z * [new branch] gh/karthickai/17/orig -> origin/gh/karthickai/17/orig 2025-12-04T09:21:04.2143082Z * [new branch] gh/karthickai/18/base -> origin/gh/karthickai/18/base 2025-12-04T09:21:04.2144567Z * [new branch] gh/karthickai/18/head -> origin/gh/karthickai/18/head 2025-12-04T09:21:04.2145885Z * [new branch] gh/karthickai/18/orig -> origin/gh/karthickai/18/orig 2025-12-04T09:21:04.2147962Z * [new branch] gh/karthickai/19/base -> origin/gh/karthickai/19/base 2025-12-04T09:21:04.2149120Z * [new branch] gh/karthickai/19/head -> origin/gh/karthickai/19/head 2025-12-04T09:21:04.2155585Z * [new branch] gh/karthickai/19/orig -> origin/gh/karthickai/19/orig 2025-12-04T09:21:04.2158162Z * [new branch] gh/karthickai/20/base -> origin/gh/karthickai/20/base 2025-12-04T09:21:04.2160317Z * [new branch] gh/karthickai/20/head -> origin/gh/karthickai/20/head 2025-12-04T09:21:04.2161658Z * [new branch] gh/karthickai/20/orig -> origin/gh/karthickai/20/orig 2025-12-04T09:21:04.2163474Z * [new branch] gh/karthickai/21/base -> origin/gh/karthickai/21/base 2025-12-04T09:21:04.2164911Z * [new branch] gh/karthickai/21/head -> origin/gh/karthickai/21/head 2025-12-04T09:21:04.2166180Z * [new branch] gh/karthickai/21/orig -> origin/gh/karthickai/21/orig 2025-12-04T09:21:04.2167979Z * [new branch] gh/karthickai/22/base -> origin/gh/karthickai/22/base 2025-12-04T09:21:04.2169170Z * [new branch] gh/karthickai/22/head -> origin/gh/karthickai/22/head 2025-12-04T09:21:04.2170365Z * [new branch] gh/karthickai/22/orig -> origin/gh/karthickai/22/orig 2025-12-04T09:21:04.2172151Z * [new branch] gh/karthickai/23/base -> origin/gh/karthickai/23/base 2025-12-04T09:21:04.2173542Z * [new branch] gh/karthickai/23/head -> origin/gh/karthickai/23/head 2025-12-04T09:21:04.2174752Z * [new branch] gh/karthickai/23/orig -> origin/gh/karthickai/23/orig 2025-12-04T09:21:04.2176356Z * [new branch] gh/karthickai/24/base -> origin/gh/karthickai/24/base 2025-12-04T09:21:04.2177673Z * [new branch] gh/karthickai/24/head -> origin/gh/karthickai/24/head 2025-12-04T09:21:04.2178872Z * [new branch] gh/karthickai/24/orig -> origin/gh/karthickai/24/orig 2025-12-04T09:21:04.2181024Z * [new branch] gh/karthickai/25/base -> origin/gh/karthickai/25/base 2025-12-04T09:21:04.2182358Z * [new branch] gh/karthickai/25/head -> origin/gh/karthickai/25/head 2025-12-04T09:21:04.2183568Z * [new branch] gh/karthickai/25/orig -> origin/gh/karthickai/25/orig 2025-12-04T09:21:04.2185077Z * [new branch] gh/karthickai/26/base -> origin/gh/karthickai/26/base 2025-12-04T09:21:04.2186472Z * [new branch] gh/karthickai/26/head -> origin/gh/karthickai/26/head 2025-12-04T09:21:04.2187993Z * [new branch] gh/karthickai/26/orig -> origin/gh/karthickai/26/orig 2025-12-04T09:21:04.2191299Z * [new branch] gh/karthickai/6/base -> origin/gh/karthickai/6/base 2025-12-04T09:21:04.2193232Z * [new branch] gh/karthickai/6/head -> origin/gh/karthickai/6/head 2025-12-04T09:21:04.2194541Z * [new branch] gh/karthickai/6/orig -> origin/gh/karthickai/6/orig 2025-12-04T09:21:04.2196635Z * [new branch] gh/krocki/1/base -> origin/gh/krocki/1/base 2025-12-04T09:21:04.2197871Z * [new branch] gh/krocki/1/head -> origin/gh/krocki/1/head 2025-12-04T09:21:04.2199181Z * [new branch] gh/krocki/1/orig -> origin/gh/krocki/1/orig 2025-12-04T09:21:04.2201336Z * [new branch] gh/krocki/2/base -> origin/gh/krocki/2/base 2025-12-04T09:21:04.2203574Z * [new branch] gh/krocki/2/head -> origin/gh/krocki/2/head 2025-12-04T09:21:04.2204805Z * [new branch] gh/krocki/2/orig -> origin/gh/krocki/2/orig 2025-12-04T09:21:04.2206910Z * [new branch] gh/kurtamohler/60/base -> origin/gh/kurtamohler/60/base 2025-12-04T09:21:04.2208105Z * [new branch] gh/kurtamohler/60/head -> origin/gh/kurtamohler/60/head 2025-12-04T09:21:04.2209335Z * [new branch] gh/kurtamohler/60/orig -> origin/gh/kurtamohler/60/orig 2025-12-04T09:21:04.2211014Z * [new branch] gh/kurtamohler/61/base -> origin/gh/kurtamohler/61/base 2025-12-04T09:21:04.2212348Z * [new branch] gh/kurtamohler/61/head -> origin/gh/kurtamohler/61/head 2025-12-04T09:21:04.2218800Z * [new branch] gh/kurtamohler/61/orig -> origin/gh/kurtamohler/61/orig 2025-12-04T09:21:04.2220914Z * [new branch] gh/kurtamohler/62/base -> origin/gh/kurtamohler/62/base 2025-12-04T09:21:04.2222166Z * [new branch] gh/kurtamohler/62/head -> origin/gh/kurtamohler/62/head 2025-12-04T09:21:04.2223406Z * [new branch] gh/kurtamohler/62/orig -> origin/gh/kurtamohler/62/orig 2025-12-04T09:21:04.2225174Z * [new branch] gh/kurtamohler/63/base -> origin/gh/kurtamohler/63/base 2025-12-04T09:21:04.2226385Z * [new branch] gh/kurtamohler/63/head -> origin/gh/kurtamohler/63/head 2025-12-04T09:21:04.2227960Z * [new branch] gh/kurtamohler/63/orig -> origin/gh/kurtamohler/63/orig 2025-12-04T09:21:04.2229644Z * [new branch] gh/kurtamohler/64/base -> origin/gh/kurtamohler/64/base 2025-12-04T09:21:04.2230821Z * [new branch] gh/kurtamohler/64/head -> origin/gh/kurtamohler/64/head 2025-12-04T09:21:04.2232048Z * [new branch] gh/kurtamohler/64/orig -> origin/gh/kurtamohler/64/orig 2025-12-04T09:21:04.2233744Z * [new branch] gh/kurtamohler/65/base -> origin/gh/kurtamohler/65/base 2025-12-04T09:21:04.2235226Z * [new branch] gh/kurtamohler/65/head -> origin/gh/kurtamohler/65/head 2025-12-04T09:21:04.2236415Z * [new branch] gh/kurtamohler/65/orig -> origin/gh/kurtamohler/65/orig 2025-12-04T09:21:04.2238077Z * [new branch] gh/kurtamohler/66/base -> origin/gh/kurtamohler/66/base 2025-12-04T09:21:04.2239426Z * [new branch] gh/kurtamohler/66/head -> origin/gh/kurtamohler/66/head 2025-12-04T09:21:04.2240677Z * [new branch] gh/kurtamohler/66/orig -> origin/gh/kurtamohler/66/orig 2025-12-04T09:21:04.2242213Z * [new branch] gh/kurtamohler/67/base -> origin/gh/kurtamohler/67/base 2025-12-04T09:21:04.2243365Z * [new branch] gh/kurtamohler/67/head -> origin/gh/kurtamohler/67/head 2025-12-04T09:21:04.2244619Z * [new branch] gh/kurtamohler/67/orig -> origin/gh/kurtamohler/67/orig 2025-12-04T09:21:04.2246580Z * [new branch] gh/kwen2501/130/base -> origin/gh/kwen2501/130/base 2025-12-04T09:21:04.2247915Z * [new branch] gh/kwen2501/130/head -> origin/gh/kwen2501/130/head 2025-12-04T09:21:04.2249142Z * [new branch] gh/kwen2501/130/orig -> origin/gh/kwen2501/130/orig 2025-12-04T09:21:04.2250788Z * [new branch] gh/kwen2501/170/base -> origin/gh/kwen2501/170/base 2025-12-04T09:21:04.2251996Z * [new branch] gh/kwen2501/170/head -> origin/gh/kwen2501/170/head 2025-12-04T09:21:04.2253808Z * [new branch] gh/kwen2501/187/base -> origin/gh/kwen2501/187/base 2025-12-04T09:21:04.2255041Z * [new branch] gh/kwen2501/187/head -> origin/gh/kwen2501/187/head 2025-12-04T09:21:04.2256328Z * [new branch] gh/kwen2501/187/orig -> origin/gh/kwen2501/187/orig 2025-12-04T09:21:04.2257960Z * [new branch] gh/kwen2501/188/base -> origin/gh/kwen2501/188/base 2025-12-04T09:21:04.2259145Z * [new branch] gh/kwen2501/188/head -> origin/gh/kwen2501/188/head 2025-12-04T09:21:04.2260325Z * [new branch] gh/kwen2501/188/orig -> origin/gh/kwen2501/188/orig 2025-12-04T09:21:04.2261897Z * [new branch] gh/kwen2501/211/base -> origin/gh/kwen2501/211/base 2025-12-04T09:21:04.2263123Z * [new branch] gh/kwen2501/211/head -> origin/gh/kwen2501/211/head 2025-12-04T09:21:04.2264652Z * [new branch] gh/kwen2501/224/base -> origin/gh/kwen2501/224/base 2025-12-04T09:21:04.2265837Z * [new branch] gh/kwen2501/224/head -> origin/gh/kwen2501/224/head 2025-12-04T09:21:04.2267271Z * [new branch] gh/kwen2501/224/orig -> origin/gh/kwen2501/224/orig 2025-12-04T09:21:04.2269103Z * [new branch] gh/kwen2501/228/base -> origin/gh/kwen2501/228/base 2025-12-04T09:21:04.2270305Z * [new branch] gh/kwen2501/228/head -> origin/gh/kwen2501/228/head 2025-12-04T09:21:04.2271526Z * [new branch] gh/kwen2501/228/orig -> origin/gh/kwen2501/228/orig 2025-12-04T09:21:04.2273329Z * [new branch] gh/kwen2501/234/base -> origin/gh/kwen2501/234/base 2025-12-04T09:21:04.2274615Z * [new branch] gh/kwen2501/234/head -> origin/gh/kwen2501/234/head 2025-12-04T09:21:04.2280149Z * [new branch] gh/kwen2501/234/orig -> origin/gh/kwen2501/234/orig 2025-12-04T09:21:04.2280411Z * [new branch] gh/kwen2501/235/base -> origin/gh/kwen2501/235/base 2025-12-04T09:21:04.2280689Z * [new branch] gh/kwen2501/235/head -> origin/gh/kwen2501/235/head 2025-12-04T09:21:04.2280943Z * [new branch] gh/kwen2501/235/orig -> origin/gh/kwen2501/235/orig 2025-12-04T09:21:04.2282030Z * [new branch] gh/kwen2501/236/base -> origin/gh/kwen2501/236/base 2025-12-04T09:21:04.2283196Z * [new branch] gh/kwen2501/236/head -> origin/gh/kwen2501/236/head 2025-12-04T09:21:04.2284913Z * [new branch] gh/kwen2501/236/orig -> origin/gh/kwen2501/236/orig 2025-12-04T09:21:04.2286658Z * [new branch] gh/kwen2501/237/base -> origin/gh/kwen2501/237/base 2025-12-04T09:21:04.2287992Z * [new branch] gh/kwen2501/237/head -> origin/gh/kwen2501/237/head 2025-12-04T09:21:04.2289174Z * [new branch] gh/kwen2501/237/orig -> origin/gh/kwen2501/237/orig 2025-12-04T09:21:04.2290752Z * [new branch] gh/kwen2501/238/base -> origin/gh/kwen2501/238/base 2025-12-04T09:21:04.2291921Z * [new branch] gh/kwen2501/238/head -> origin/gh/kwen2501/238/head 2025-12-04T09:21:04.2293189Z * [new branch] gh/kwen2501/238/orig -> origin/gh/kwen2501/238/orig 2025-12-04T09:21:04.2294801Z * [new branch] gh/kwen2501/240/base -> origin/gh/kwen2501/240/base 2025-12-04T09:21:04.2295934Z * [new branch] gh/kwen2501/240/head -> origin/gh/kwen2501/240/head 2025-12-04T09:21:04.2297079Z * [new branch] gh/kwen2501/240/orig -> origin/gh/kwen2501/240/orig 2025-12-04T09:21:04.2298655Z * [new branch] gh/kwen2501/241/base -> origin/gh/kwen2501/241/base 2025-12-04T09:21:04.2299833Z * [new branch] gh/kwen2501/241/head -> origin/gh/kwen2501/241/head 2025-12-04T09:21:04.2301630Z * [new branch] gh/kwen2501/241/orig -> origin/gh/kwen2501/241/orig 2025-12-04T09:21:04.2303595Z * [new branch] gh/kwen2501/247/base -> origin/gh/kwen2501/247/base 2025-12-04T09:21:04.2304772Z * [new branch] gh/kwen2501/247/head -> origin/gh/kwen2501/247/head 2025-12-04T09:21:04.2306033Z * [new branch] gh/kwen2501/247/orig -> origin/gh/kwen2501/247/orig 2025-12-04T09:21:04.2307839Z * [new branch] gh/kwen2501/252/base -> origin/gh/kwen2501/252/base 2025-12-04T09:21:04.2309027Z * [new branch] gh/kwen2501/252/head -> origin/gh/kwen2501/252/head 2025-12-04T09:21:04.2310274Z * [new branch] gh/kwen2501/252/orig -> origin/gh/kwen2501/252/orig 2025-12-04T09:21:04.2312407Z * [new branch] gh/kwen2501/259/base -> origin/gh/kwen2501/259/base 2025-12-04T09:21:04.2313749Z * [new branch] gh/kwen2501/259/head -> origin/gh/kwen2501/259/head 2025-12-04T09:21:04.2314983Z * [new branch] gh/kwen2501/259/orig -> origin/gh/kwen2501/259/orig 2025-12-04T09:21:04.2316865Z * [new branch] gh/kwen2501/260/base -> origin/gh/kwen2501/260/base 2025-12-04T09:21:04.2318318Z * [new branch] gh/kwen2501/260/head -> origin/gh/kwen2501/260/head 2025-12-04T09:21:04.2319523Z * [new branch] gh/kwen2501/260/orig -> origin/gh/kwen2501/260/orig 2025-12-04T09:21:04.2321284Z * [new branch] gh/kwen2501/268/base -> origin/gh/kwen2501/268/base 2025-12-04T09:21:04.2322504Z * [new branch] gh/kwen2501/268/head -> origin/gh/kwen2501/268/head 2025-12-04T09:21:04.2323762Z * [new branch] gh/kwen2501/268/orig -> origin/gh/kwen2501/268/orig 2025-12-04T09:21:04.2325380Z * [new branch] gh/kwen2501/269/base -> origin/gh/kwen2501/269/base 2025-12-04T09:21:04.2326708Z * [new branch] gh/kwen2501/269/head -> origin/gh/kwen2501/269/head 2025-12-04T09:21:04.2327893Z * [new branch] gh/kwen2501/269/orig -> origin/gh/kwen2501/269/orig 2025-12-04T09:21:04.2329603Z * [new branch] gh/kwen2501/270/base -> origin/gh/kwen2501/270/base 2025-12-04T09:21:04.2330961Z * [new branch] gh/kwen2501/270/head -> origin/gh/kwen2501/270/head 2025-12-04T09:21:04.2332153Z * [new branch] gh/kwen2501/270/orig -> origin/gh/kwen2501/270/orig 2025-12-04T09:21:04.2333859Z * [new branch] gh/kwen2501/271/base -> origin/gh/kwen2501/271/base 2025-12-04T09:21:04.2335086Z * [new branch] gh/kwen2501/271/head -> origin/gh/kwen2501/271/head 2025-12-04T09:21:04.2336355Z * [new branch] gh/kwen2501/271/orig -> origin/gh/kwen2501/271/orig 2025-12-04T09:21:04.2342520Z * [new branch] gh/kwen2501/274/base -> origin/gh/kwen2501/274/base 2025-12-04T09:21:04.2342777Z * [new branch] gh/kwen2501/274/head -> origin/gh/kwen2501/274/head 2025-12-04T09:21:04.2343251Z * [new branch] gh/kwen2501/274/orig -> origin/gh/kwen2501/274/orig 2025-12-04T09:21:04.2343689Z * [new branch] gh/kwen2501/275/base -> origin/gh/kwen2501/275/base 2025-12-04T09:21:04.2344952Z * [new branch] gh/kwen2501/275/head -> origin/gh/kwen2501/275/head 2025-12-04T09:21:04.2347015Z * [new branch] gh/kwen2501/275/orig -> origin/gh/kwen2501/275/orig 2025-12-04T09:21:04.2348756Z * [new branch] gh/kwen2501/276/base -> origin/gh/kwen2501/276/base 2025-12-04T09:21:04.2349912Z * [new branch] gh/kwen2501/276/head -> origin/gh/kwen2501/276/head 2025-12-04T09:21:04.2351134Z * [new branch] gh/kwen2501/276/orig -> origin/gh/kwen2501/276/orig 2025-12-04T09:21:04.2352809Z * [new branch] gh/kwen2501/277/base -> origin/gh/kwen2501/277/base 2025-12-04T09:21:04.2354063Z * [new branch] gh/kwen2501/277/head -> origin/gh/kwen2501/277/head 2025-12-04T09:21:04.2355250Z * [new branch] gh/kwen2501/277/orig -> origin/gh/kwen2501/277/orig 2025-12-04T09:21:04.2357055Z * [new branch] gh/kwen2501/278/base -> origin/gh/kwen2501/278/base 2025-12-04T09:21:04.2358355Z * [new branch] gh/kwen2501/278/head -> origin/gh/kwen2501/278/head 2025-12-04T09:21:04.2359561Z * [new branch] gh/kwen2501/278/orig -> origin/gh/kwen2501/278/orig 2025-12-04T09:21:04.2361282Z * [new branch] gh/kwen2501/279/base -> origin/gh/kwen2501/279/base 2025-12-04T09:21:04.2362612Z * [new branch] gh/kwen2501/279/head -> origin/gh/kwen2501/279/head 2025-12-04T09:21:04.2363864Z * [new branch] gh/kwen2501/279/orig -> origin/gh/kwen2501/279/orig 2025-12-04T09:21:04.2365548Z * [new branch] gh/kwen2501/280/base -> origin/gh/kwen2501/280/base 2025-12-04T09:21:04.2366860Z * [new branch] gh/kwen2501/280/head -> origin/gh/kwen2501/280/head 2025-12-04T09:21:04.2368085Z * [new branch] gh/kwen2501/280/orig -> origin/gh/kwen2501/280/orig 2025-12-04T09:21:04.2369716Z * [new branch] gh/kwen2501/281/base -> origin/gh/kwen2501/281/base 2025-12-04T09:21:04.2370904Z * [new branch] gh/kwen2501/281/head -> origin/gh/kwen2501/281/head 2025-12-04T09:21:04.2372077Z * [new branch] gh/kwen2501/281/orig -> origin/gh/kwen2501/281/orig 2025-12-04T09:21:04.2373840Z * [new branch] gh/kwen2501/282/base -> origin/gh/kwen2501/282/base 2025-12-04T09:21:04.2375129Z * [new branch] gh/kwen2501/282/head -> origin/gh/kwen2501/282/head 2025-12-04T09:21:04.2376317Z * [new branch] gh/kwen2501/282/orig -> origin/gh/kwen2501/282/orig 2025-12-04T09:21:04.2377960Z * [new branch] gh/kwen2501/283/base -> origin/gh/kwen2501/283/base 2025-12-04T09:21:04.2379231Z * [new branch] gh/kwen2501/283/head -> origin/gh/kwen2501/283/head 2025-12-04T09:21:04.2380429Z * [new branch] gh/kwen2501/283/orig -> origin/gh/kwen2501/283/orig 2025-12-04T09:21:04.2382113Z * [new branch] gh/kwen2501/284/base -> origin/gh/kwen2501/284/base 2025-12-04T09:21:04.2383403Z * [new branch] gh/kwen2501/284/head -> origin/gh/kwen2501/284/head 2025-12-04T09:21:04.2384622Z * [new branch] gh/kwen2501/284/orig -> origin/gh/kwen2501/284/orig 2025-12-04T09:21:04.2386202Z * [new branch] gh/kwen2501/285/base -> origin/gh/kwen2501/285/base 2025-12-04T09:21:04.2387907Z * [new branch] gh/kwen2501/285/head -> origin/gh/kwen2501/285/head 2025-12-04T09:21:04.2389102Z * [new branch] gh/kwen2501/285/orig -> origin/gh/kwen2501/285/orig 2025-12-04T09:21:04.2390885Z * [new branch] gh/kwen2501/286/base -> origin/gh/kwen2501/286/base 2025-12-04T09:21:04.2392204Z * [new branch] gh/kwen2501/286/head -> origin/gh/kwen2501/286/head 2025-12-04T09:21:04.2393476Z * [new branch] gh/kwen2501/286/orig -> origin/gh/kwen2501/286/orig 2025-12-04T09:21:04.2395405Z * [new branch] gh/kwen2501/287/base -> origin/gh/kwen2501/287/base 2025-12-04T09:21:04.2396695Z * [new branch] gh/kwen2501/287/head -> origin/gh/kwen2501/287/head 2025-12-04T09:21:04.2398131Z * [new branch] gh/kwen2501/287/orig -> origin/gh/kwen2501/287/orig 2025-12-04T09:21:04.2399653Z * [new branch] gh/kwen2501/288/base -> origin/gh/kwen2501/288/base 2025-12-04T09:21:04.2408546Z * [new branch] gh/kwen2501/288/head -> origin/gh/kwen2501/288/head 2025-12-04T09:21:04.2409734Z * [new branch] gh/kwen2501/288/orig -> origin/gh/kwen2501/288/orig 2025-12-04T09:21:04.2411794Z * [new branch] gh/laithsakka/251/base -> origin/gh/laithsakka/251/base 2025-12-04T09:21:04.2413072Z * [new branch] gh/laithsakka/251/head -> origin/gh/laithsakka/251/head 2025-12-04T09:21:04.2414297Z * [new branch] gh/laithsakka/251/orig -> origin/gh/laithsakka/251/orig 2025-12-04T09:21:04.2416113Z * [new branch] gh/laithsakka/276/base -> origin/gh/laithsakka/276/base 2025-12-04T09:21:04.2417296Z * [new branch] gh/laithsakka/276/head -> origin/gh/laithsakka/276/head 2025-12-04T09:21:04.2418476Z * [new branch] gh/laithsakka/276/orig -> origin/gh/laithsakka/276/orig 2025-12-04T09:21:04.2420157Z * [new branch] gh/laithsakka/28/base -> origin/gh/laithsakka/28/base 2025-12-04T09:21:04.2421616Z * [new branch] gh/laithsakka/29/base -> origin/gh/laithsakka/29/base 2025-12-04T09:21:04.2423062Z * [new branch] gh/laithsakka/30/base -> origin/gh/laithsakka/30/base 2025-12-04T09:21:04.2424335Z * [new branch] gh/laithsakka/30/head -> origin/gh/laithsakka/30/head 2025-12-04T09:21:04.2425792Z * [new branch] gh/laithsakka/31/base -> origin/gh/laithsakka/31/base 2025-12-04T09:21:04.2427033Z * [new branch] gh/laithsakka/31/head -> origin/gh/laithsakka/31/head 2025-12-04T09:21:04.2429119Z * [new branch] gh/laithsakka/313/base -> origin/gh/laithsakka/313/base 2025-12-04T09:21:04.2430442Z * [new branch] gh/laithsakka/313/head -> origin/gh/laithsakka/313/head 2025-12-04T09:21:04.2431691Z * [new branch] gh/laithsakka/313/orig -> origin/gh/laithsakka/313/orig 2025-12-04T09:21:04.2433608Z * [new branch] gh/laithsakka/316/base -> origin/gh/laithsakka/316/base 2025-12-04T09:21:04.2434812Z * [new branch] gh/laithsakka/316/head -> origin/gh/laithsakka/316/head 2025-12-04T09:21:04.2436026Z * [new branch] gh/laithsakka/316/orig -> origin/gh/laithsakka/316/orig 2025-12-04T09:21:04.2437723Z * [new branch] gh/laithsakka/317/base -> origin/gh/laithsakka/317/base 2025-12-04T09:21:04.2439362Z * [new branch] gh/laithsakka/317/head -> origin/gh/laithsakka/317/head 2025-12-04T09:21:04.2440661Z * [new branch] gh/laithsakka/317/orig -> origin/gh/laithsakka/317/orig 2025-12-04T09:21:04.2442374Z * [new branch] gh/laithsakka/319/base -> origin/gh/laithsakka/319/base 2025-12-04T09:21:04.2443589Z * [new branch] gh/laithsakka/319/head -> origin/gh/laithsakka/319/head 2025-12-04T09:21:04.2444801Z * [new branch] gh/laithsakka/319/orig -> origin/gh/laithsakka/319/orig 2025-12-04T09:21:04.2446244Z * [new branch] gh/laithsakka/32/base -> origin/gh/laithsakka/32/base 2025-12-04T09:21:04.2447453Z * [new branch] gh/laithsakka/32/head -> origin/gh/laithsakka/32/head 2025-12-04T09:21:04.2460577Z * [new branch] gh/laithsakka/320/base -> origin/gh/laithsakka/320/base 2025-12-04T09:21:04.2461100Z * [new branch] gh/laithsakka/320/head -> origin/gh/laithsakka/320/head 2025-12-04T09:21:04.2461395Z * [new branch] gh/laithsakka/320/orig -> origin/gh/laithsakka/320/orig 2025-12-04T09:21:04.2461766Z * [new branch] gh/laithsakka/321/base -> origin/gh/laithsakka/321/base 2025-12-04T09:21:04.2462184Z * [new branch] gh/laithsakka/321/head -> origin/gh/laithsakka/321/head 2025-12-04T09:21:04.2467730Z * [new branch] gh/laithsakka/321/orig -> origin/gh/laithsakka/321/orig 2025-12-04T09:21:04.2468056Z * [new branch] gh/laithsakka/322/base -> origin/gh/laithsakka/322/base 2025-12-04T09:21:04.2468369Z * [new branch] gh/laithsakka/322/head -> origin/gh/laithsakka/322/head 2025-12-04T09:21:04.2468642Z * [new branch] gh/laithsakka/322/orig -> origin/gh/laithsakka/322/orig 2025-12-04T09:21:04.2468923Z * [new branch] gh/laithsakka/323/base -> origin/gh/laithsakka/323/base 2025-12-04T09:21:04.2469220Z * [new branch] gh/laithsakka/323/head -> origin/gh/laithsakka/323/head 2025-12-04T09:21:04.2469763Z * [new branch] gh/laithsakka/323/orig -> origin/gh/laithsakka/323/orig 2025-12-04T09:21:04.2471610Z * [new branch] gh/laithsakka/324/base -> origin/gh/laithsakka/324/base 2025-12-04T09:21:04.2472753Z * [new branch] gh/laithsakka/324/head -> origin/gh/laithsakka/324/head 2025-12-04T09:21:04.2474008Z * [new branch] gh/laithsakka/324/orig -> origin/gh/laithsakka/324/orig 2025-12-04T09:21:04.2475708Z * [new branch] gh/laithsakka/325/base -> origin/gh/laithsakka/325/base 2025-12-04T09:21:04.2476928Z * [new branch] gh/laithsakka/325/head -> origin/gh/laithsakka/325/head 2025-12-04T09:21:04.2478150Z * [new branch] gh/laithsakka/325/orig -> origin/gh/laithsakka/325/orig 2025-12-04T09:21:04.2480157Z * [new branch] gh/laithsakka/326/base -> origin/gh/laithsakka/326/base 2025-12-04T09:21:04.2481409Z * [new branch] gh/laithsakka/326/head -> origin/gh/laithsakka/326/head 2025-12-04T09:21:04.2482582Z * [new branch] gh/laithsakka/326/orig -> origin/gh/laithsakka/326/orig 2025-12-04T09:21:04.2484243Z * [new branch] gh/laithsakka/327/base -> origin/gh/laithsakka/327/base 2025-12-04T09:21:04.2485537Z * [new branch] gh/laithsakka/327/head -> origin/gh/laithsakka/327/head 2025-12-04T09:21:04.2486870Z * [new branch] gh/laithsakka/327/orig -> origin/gh/laithsakka/327/orig 2025-12-04T09:21:04.2488500Z * [new branch] gh/laithsakka/328/base -> origin/gh/laithsakka/328/base 2025-12-04T09:21:04.2489681Z * [new branch] gh/laithsakka/328/head -> origin/gh/laithsakka/328/head 2025-12-04T09:21:04.2490885Z * [new branch] gh/laithsakka/328/orig -> origin/gh/laithsakka/328/orig 2025-12-04T09:21:04.2493198Z * [new branch] gh/liangel/4/base -> origin/gh/liangel/4/base 2025-12-04T09:21:04.2494416Z * [new branch] gh/liangel/4/head -> origin/gh/liangel/4/head 2025-12-04T09:21:04.2495602Z * [new branch] gh/liangel/4/orig -> origin/gh/liangel/4/orig 2025-12-04T09:21:04.2499992Z * [new branch] gh/lucaskabela/1/base -> origin/gh/lucaskabela/1/base 2025-12-04T09:21:04.2501938Z * [new branch] gh/lucaskabela/1/head -> origin/gh/lucaskabela/1/head 2025-12-04T09:21:04.2504046Z * [new branch] gh/lw/4/base -> origin/gh/lw/4/base 2025-12-04T09:21:04.2505334Z * [new branch] gh/lw/4/head -> origin/gh/lw/4/head 2025-12-04T09:21:04.2506651Z * [new branch] gh/lw/4/orig -> origin/gh/lw/4/orig 2025-12-04T09:21:04.2508388Z * [new branch] gh/lw/5/base -> origin/gh/lw/5/base 2025-12-04T09:21:04.2509661Z * [new branch] gh/lw/5/head -> origin/gh/lw/5/head 2025-12-04T09:21:04.2510841Z * [new branch] gh/lw/5/orig -> origin/gh/lw/5/orig 2025-12-04T09:21:04.2512454Z * [new branch] gh/lw/6/base -> origin/gh/lw/6/base 2025-12-04T09:21:04.2513768Z * [new branch] gh/lw/6/head -> origin/gh/lw/6/head 2025-12-04T09:21:04.2514947Z * [new branch] gh/lw/6/orig -> origin/gh/lw/6/orig 2025-12-04T09:21:04.2516848Z * [new branch] gh/malfet/14/base -> origin/gh/malfet/14/base 2025-12-04T09:21:04.2518595Z * [new branch] gh/malfet/417/base -> origin/gh/malfet/417/base 2025-12-04T09:21:04.2519901Z * [new branch] gh/malfet/417/head -> origin/gh/malfet/417/head 2025-12-04T09:21:04.2521146Z * [new branch] gh/malfet/417/orig -> origin/gh/malfet/417/orig 2025-12-04T09:21:04.2522731Z * [new branch] gh/malfet/506/base -> origin/gh/malfet/506/base 2025-12-04T09:21:04.2523927Z * [new branch] gh/malfet/506/head -> origin/gh/malfet/506/head 2025-12-04T09:21:04.2530031Z * [new branch] gh/malfet/506/orig -> origin/gh/malfet/506/orig 2025-12-04T09:21:04.2530303Z * [new branch] gh/malfet/517/base -> origin/gh/malfet/517/base 2025-12-04T09:21:04.2530552Z * [new branch] gh/malfet/517/head -> origin/gh/malfet/517/head 2025-12-04T09:21:04.2530810Z * [new branch] gh/malfet/528/base -> origin/gh/malfet/528/base 2025-12-04T09:21:04.2531346Z * [new branch] gh/malfet/528/head -> origin/gh/malfet/528/head 2025-12-04T09:21:04.2532593Z * [new branch] gh/malfet/528/orig -> origin/gh/malfet/528/orig 2025-12-04T09:21:04.2534197Z * [new branch] gh/malfet/537/base -> origin/gh/malfet/537/base 2025-12-04T09:21:04.2535394Z * [new branch] gh/malfet/537/head -> origin/gh/malfet/537/head 2025-12-04T09:21:04.2536754Z * [new branch] gh/malfet/537/orig -> origin/gh/malfet/537/orig 2025-12-04T09:21:04.2538455Z * [new branch] gh/malfet/546/base -> origin/gh/malfet/546/base 2025-12-04T09:21:04.2539661Z * [new branch] gh/malfet/546/head -> origin/gh/malfet/546/head 2025-12-04T09:21:04.2540875Z * [new branch] gh/malfet/546/orig -> origin/gh/malfet/546/orig 2025-12-04T09:21:04.2542481Z * [new branch] gh/malfet/565/base -> origin/gh/malfet/565/base 2025-12-04T09:21:04.2543714Z * [new branch] gh/malfet/565/head -> origin/gh/malfet/565/head 2025-12-04T09:21:04.2544916Z * [new branch] gh/malfet/565/orig -> origin/gh/malfet/565/orig 2025-12-04T09:21:04.2546655Z * [new branch] gh/malfet/575/base -> origin/gh/malfet/575/base 2025-12-04T09:21:04.2548097Z * [new branch] gh/malfet/575/head -> origin/gh/malfet/575/head 2025-12-04T09:21:04.2549323Z * [new branch] gh/malfet/575/orig -> origin/gh/malfet/575/orig 2025-12-04T09:21:04.2550969Z * [new branch] gh/malfet/580/base -> origin/gh/malfet/580/base 2025-12-04T09:21:04.2552210Z * [new branch] gh/malfet/580/head -> origin/gh/malfet/580/head 2025-12-04T09:21:04.2553539Z * [new branch] gh/malfet/580/orig -> origin/gh/malfet/580/orig 2025-12-04T09:21:04.2555150Z * [new branch] gh/malfet/581/base -> origin/gh/malfet/581/base 2025-12-04T09:21:04.2556379Z * [new branch] gh/malfet/581/head -> origin/gh/malfet/581/head 2025-12-04T09:21:04.2557569Z * [new branch] gh/malfet/581/orig -> origin/gh/malfet/581/orig 2025-12-04T09:21:04.2559206Z * [new branch] gh/malfet/583/base -> origin/gh/malfet/583/base 2025-12-04T09:21:04.2560424Z * [new branch] gh/malfet/583/head -> origin/gh/malfet/583/head 2025-12-04T09:21:04.2561620Z * [new branch] gh/malfet/583/orig -> origin/gh/malfet/583/orig 2025-12-04T09:21:04.2563192Z * [new branch] gh/malfet/586/base -> origin/gh/malfet/586/base 2025-12-04T09:21:04.2564428Z * [new branch] gh/malfet/586/head -> origin/gh/malfet/586/head 2025-12-04T09:21:04.2565539Z * [new branch] gh/malfet/586/orig -> origin/gh/malfet/586/orig 2025-12-04T09:21:04.2567448Z * [new branch] gh/malfet/587/base -> origin/gh/malfet/587/base 2025-12-04T09:21:04.2568654Z * [new branch] gh/malfet/587/head -> origin/gh/malfet/587/head 2025-12-04T09:21:04.2570056Z * [new branch] gh/malfet/587/orig -> origin/gh/malfet/587/orig 2025-12-04T09:21:04.2571508Z * [new branch] gh/malfet/588/base -> origin/gh/malfet/588/base 2025-12-04T09:21:04.2572697Z * [new branch] gh/malfet/588/head -> origin/gh/malfet/588/head 2025-12-04T09:21:04.2574039Z * [new branch] gh/malfet/588/orig -> origin/gh/malfet/588/orig 2025-12-04T09:21:04.2575656Z * [new branch] gh/malfet/589/base -> origin/gh/malfet/589/base 2025-12-04T09:21:04.2576887Z * [new branch] gh/malfet/589/head -> origin/gh/malfet/589/head 2025-12-04T09:21:04.2578091Z * [new branch] gh/malfet/589/orig -> origin/gh/malfet/589/orig 2025-12-04T09:21:04.2580071Z * [new branch] gh/malfet/590/base -> origin/gh/malfet/590/base 2025-12-04T09:21:04.2581257Z * [new branch] gh/malfet/590/head -> origin/gh/malfet/590/head 2025-12-04T09:21:04.2582441Z * [new branch] gh/malfet/590/orig -> origin/gh/malfet/590/orig 2025-12-04T09:21:04.2584604Z * [new branch] gh/malfet/591/base -> origin/gh/malfet/591/base 2025-12-04T09:21:04.2585804Z * [new branch] gh/malfet/591/head -> origin/gh/malfet/591/head 2025-12-04T09:21:04.2587273Z * [new branch] gh/malfet/591/orig -> origin/gh/malfet/591/orig 2025-12-04T09:21:04.2592533Z * [new branch] gh/malfet/592/base -> origin/gh/malfet/592/base 2025-12-04T09:21:04.2593875Z * [new branch] gh/malfet/592/head -> origin/gh/malfet/592/head 2025-12-04T09:21:04.2595057Z * [new branch] gh/malfet/592/orig -> origin/gh/malfet/592/orig 2025-12-04T09:21:04.2596712Z * [new branch] gh/malfet/593/base -> origin/gh/malfet/593/base 2025-12-04T09:21:04.2597910Z * [new branch] gh/malfet/593/head -> origin/gh/malfet/593/head 2025-12-04T09:21:04.2599220Z * [new branch] gh/malfet/593/orig -> origin/gh/malfet/593/orig 2025-12-04T09:21:04.2601262Z * [new branch] gh/malfet/594/base -> origin/gh/malfet/594/base 2025-12-04T09:21:04.2602773Z * [new branch] gh/malfet/594/head -> origin/gh/malfet/594/head 2025-12-04T09:21:04.2603963Z * [new branch] gh/malfet/594/orig -> origin/gh/malfet/594/orig 2025-12-04T09:21:04.2605621Z * [new branch] gh/malfet/595/base -> origin/gh/malfet/595/base 2025-12-04T09:21:04.2606837Z * [new branch] gh/malfet/595/head -> origin/gh/malfet/595/head 2025-12-04T09:21:04.2608215Z * [new branch] gh/malfet/595/orig -> origin/gh/malfet/595/orig 2025-12-04T09:21:04.2609983Z * [new branch] gh/malfet/596/base -> origin/gh/malfet/596/base 2025-12-04T09:21:04.2611229Z * [new branch] gh/malfet/596/head -> origin/gh/malfet/596/head 2025-12-04T09:21:04.2612459Z * [new branch] gh/malfet/596/orig -> origin/gh/malfet/596/orig 2025-12-04T09:21:04.2614745Z * [new branch] gh/malfet/597/base -> origin/gh/malfet/597/base 2025-12-04T09:21:04.2615939Z * [new branch] gh/malfet/597/head -> origin/gh/malfet/597/head 2025-12-04T09:21:04.2617161Z * [new branch] gh/malfet/597/orig -> origin/gh/malfet/597/orig 2025-12-04T09:21:04.2618730Z * [new branch] gh/malfet/598/base -> origin/gh/malfet/598/base 2025-12-04T09:21:04.2620191Z * [new branch] gh/malfet/598/head -> origin/gh/malfet/598/head 2025-12-04T09:21:04.2621337Z * [new branch] gh/malfet/598/orig -> origin/gh/malfet/598/orig 2025-12-04T09:21:04.2622816Z * [new branch] gh/malfet/599/base -> origin/gh/malfet/599/base 2025-12-04T09:21:04.2623972Z * [new branch] gh/malfet/599/head -> origin/gh/malfet/599/head 2025-12-04T09:21:04.2625291Z * [new branch] gh/malfet/599/orig -> origin/gh/malfet/599/orig 2025-12-04T09:21:04.2627201Z * [new branch] gh/malfet/600/base -> origin/gh/malfet/600/base 2025-12-04T09:21:04.2628464Z * [new branch] gh/malfet/600/head -> origin/gh/malfet/600/head 2025-12-04T09:21:04.2629689Z * [new branch] gh/malfet/600/orig -> origin/gh/malfet/600/orig 2025-12-04T09:21:04.2631624Z * [new branch] gh/malfet/601/base -> origin/gh/malfet/601/base 2025-12-04T09:21:04.2632879Z * [new branch] gh/malfet/601/head -> origin/gh/malfet/601/head 2025-12-04T09:21:04.2634119Z * [new branch] gh/malfet/601/orig -> origin/gh/malfet/601/orig 2025-12-04T09:21:04.2635840Z * [new branch] gh/malfet/602/base -> origin/gh/malfet/602/base 2025-12-04T09:21:04.2637041Z * [new branch] gh/malfet/602/head -> origin/gh/malfet/602/head 2025-12-04T09:21:04.2638262Z * [new branch] gh/malfet/602/orig -> origin/gh/malfet/602/orig 2025-12-04T09:21:04.2639943Z * [new branch] gh/malfet/603/base -> origin/gh/malfet/603/base 2025-12-04T09:21:04.2641040Z * [new branch] gh/malfet/603/head -> origin/gh/malfet/603/head 2025-12-04T09:21:04.2642320Z * [new branch] gh/malfet/603/orig -> origin/gh/malfet/603/orig 2025-12-04T09:21:04.2643989Z * [new branch] gh/malfet/604/base -> origin/gh/malfet/604/base 2025-12-04T09:21:04.2645161Z * [new branch] gh/malfet/604/head -> origin/gh/malfet/604/head 2025-12-04T09:21:04.2646337Z * [new branch] gh/malfet/604/orig -> origin/gh/malfet/604/orig 2025-12-04T09:21:04.2647960Z * [new branch] gh/malfet/605/base -> origin/gh/malfet/605/base 2025-12-04T09:21:04.2649143Z * [new branch] gh/malfet/605/head -> origin/gh/malfet/605/head 2025-12-04T09:21:04.2655651Z * [new branch] gh/malfet/605/orig -> origin/gh/malfet/605/orig 2025-12-04T09:21:04.2657392Z * [new branch] gh/malfet/606/base -> origin/gh/malfet/606/base 2025-12-04T09:21:04.2658797Z * [new branch] gh/malfet/606/head -> origin/gh/malfet/606/head 2025-12-04T09:21:04.2660025Z * [new branch] gh/malfet/606/orig -> origin/gh/malfet/606/orig 2025-12-04T09:21:04.2661739Z * [new branch] gh/malfet/607/base -> origin/gh/malfet/607/base 2025-12-04T09:21:04.2663045Z * [new branch] gh/malfet/607/head -> origin/gh/malfet/607/head 2025-12-04T09:21:04.2664460Z * [new branch] gh/malfet/607/orig -> origin/gh/malfet/607/orig 2025-12-04T09:21:04.2666069Z * [new branch] gh/malfet/608/base -> origin/gh/malfet/608/base 2025-12-04T09:21:04.2667654Z * [new branch] gh/malfet/608/head -> origin/gh/malfet/608/head 2025-12-04T09:21:04.2668897Z * [new branch] gh/malfet/608/orig -> origin/gh/malfet/608/orig 2025-12-04T09:21:04.2670604Z * [new branch] gh/malfet/609/base -> origin/gh/malfet/609/base 2025-12-04T09:21:04.2671833Z * [new branch] gh/malfet/609/head -> origin/gh/malfet/609/head 2025-12-04T09:21:04.2673043Z * [new branch] gh/malfet/609/orig -> origin/gh/malfet/609/orig 2025-12-04T09:21:04.2674803Z * [new branch] gh/malfet/610/base -> origin/gh/malfet/610/base 2025-12-04T09:21:04.2676097Z * [new branch] gh/malfet/610/head -> origin/gh/malfet/610/head 2025-12-04T09:21:04.2677377Z * [new branch] gh/malfet/610/orig -> origin/gh/malfet/610/orig 2025-12-04T09:21:04.2679164Z * [new branch] gh/malfet/611/base -> origin/gh/malfet/611/base 2025-12-04T09:21:04.2680355Z * [new branch] gh/malfet/611/head -> origin/gh/malfet/611/head 2025-12-04T09:21:04.2682336Z * [new branch] gh/malfet/611/orig -> origin/gh/malfet/611/orig 2025-12-04T09:21:04.2683856Z * [new branch] gh/malfet/612/base -> origin/gh/malfet/612/base 2025-12-04T09:21:04.2685057Z * [new branch] gh/malfet/612/head -> origin/gh/malfet/612/head 2025-12-04T09:21:04.2686339Z * [new branch] gh/malfet/612/orig -> origin/gh/malfet/612/orig 2025-12-04T09:21:04.2688063Z * [new branch] gh/malfet/64/base -> origin/gh/malfet/64/base 2025-12-04T09:21:04.2689234Z * [new branch] gh/malfet/64/head -> origin/gh/malfet/64/head 2025-12-04T09:21:04.2691177Z * [new branch] gh/manuelcandales/11/base -> origin/gh/manuelcandales/11/base 2025-12-04T09:21:04.2692360Z * [new branch] gh/manuelcandales/11/head -> origin/gh/manuelcandales/11/head 2025-12-04T09:21:04.2693535Z * [new branch] gh/manuelcandales/11/orig -> origin/gh/manuelcandales/11/orig 2025-12-04T09:21:04.2695678Z * [new branch] gh/markkm/1/base -> origin/gh/markkm/1/base 2025-12-04T09:21:04.2697668Z * [new branch] gh/masnesral/1/base -> origin/gh/masnesral/1/base 2025-12-04T09:21:04.2698872Z * [new branch] gh/masnesral/1/head -> origin/gh/masnesral/1/head 2025-12-04T09:21:04.2700085Z * [new branch] gh/masnesral/1/orig -> origin/gh/masnesral/1/orig 2025-12-04T09:21:04.2705287Z * [new branch] gh/mhorowitz/0/base -> origin/gh/mhorowitz/0/base 2025-12-04T09:21:04.2706515Z * [new branch] gh/mhorowitz/0/head -> origin/gh/mhorowitz/0/head 2025-12-04T09:21:04.2708183Z * [new branch] gh/mhorowitz/1/base -> origin/gh/mhorowitz/1/base 2025-12-04T09:21:04.2709485Z * [new branch] gh/mhorowitz/1/head -> origin/gh/mhorowitz/1/head 2025-12-04T09:21:04.2710990Z * [new branch] gh/mhorowitz/2/base -> origin/gh/mhorowitz/2/base 2025-12-04T09:21:04.2712280Z * [new branch] gh/mhorowitz/2/head -> origin/gh/mhorowitz/2/head 2025-12-04T09:21:04.2719025Z * [new branch] gh/mhorowitz/3/base -> origin/gh/mhorowitz/3/base 2025-12-04T09:21:04.2720167Z * [new branch] gh/mhorowitz/3/head -> origin/gh/mhorowitz/3/head 2025-12-04T09:21:04.2721720Z * [new branch] gh/mhorowitz/4/base -> origin/gh/mhorowitz/4/base 2025-12-04T09:21:04.2722843Z * [new branch] gh/mhorowitz/4/head -> origin/gh/mhorowitz/4/head 2025-12-04T09:21:04.2724971Z * [new branch] gh/mhorowitz/5/base -> origin/gh/mhorowitz/5/base 2025-12-04T09:21:04.2726111Z * [new branch] gh/mhorowitz/5/head -> origin/gh/mhorowitz/5/head 2025-12-04T09:21:04.2727626Z * [new branch] gh/mhorowitz/6/base -> origin/gh/mhorowitz/6/base 2025-12-04T09:21:04.2728755Z * [new branch] gh/mhorowitz/6/head -> origin/gh/mhorowitz/6/head 2025-12-04T09:21:04.2730752Z * [new branch] gh/mikaylagawarecki/234/base -> origin/gh/mikaylagawarecki/234/base 2025-12-04T09:21:04.2732028Z * [new branch] gh/mikaylagawarecki/234/head -> origin/gh/mikaylagawarecki/234/head 2025-12-04T09:21:04.2733658Z * [new branch] gh/mikaylagawarecki/235/base -> origin/gh/mikaylagawarecki/235/base 2025-12-04T09:21:04.2734885Z * [new branch] gh/mikaylagawarecki/235/head -> origin/gh/mikaylagawarecki/235/head 2025-12-04T09:21:04.2736299Z * [new branch] gh/mikaylagawarecki/236/base -> origin/gh/mikaylagawarecki/236/base 2025-12-04T09:21:04.2737425Z * [new branch] gh/mikaylagawarecki/236/head -> origin/gh/mikaylagawarecki/236/head 2025-12-04T09:21:04.2738950Z * [new branch] gh/mikaylagawarecki/237/base -> origin/gh/mikaylagawarecki/237/base 2025-12-04T09:21:04.2740031Z * [new branch] gh/mikaylagawarecki/237/head -> origin/gh/mikaylagawarecki/237/head 2025-12-04T09:21:04.2741686Z * [new branch] gh/mikaylagawarecki/238/base -> origin/gh/mikaylagawarecki/238/base 2025-12-04T09:21:04.2742850Z * [new branch] gh/mikaylagawarecki/238/head -> origin/gh/mikaylagawarecki/238/head 2025-12-04T09:21:04.2744492Z * [new branch] gh/mikaylagawarecki/336/base -> origin/gh/mikaylagawarecki/336/base 2025-12-04T09:21:04.2745710Z * [new branch] gh/mikaylagawarecki/336/head -> origin/gh/mikaylagawarecki/336/head 2025-12-04T09:21:04.2747199Z * [new branch] gh/mikaylagawarecki/336/orig -> origin/gh/mikaylagawarecki/336/orig 2025-12-04T09:21:04.2749238Z * [new branch] gh/mikaylagawarecki/341/base -> origin/gh/mikaylagawarecki/341/base 2025-12-04T09:21:04.2750390Z * [new branch] gh/mikaylagawarecki/341/head -> origin/gh/mikaylagawarecki/341/head 2025-12-04T09:21:04.2751610Z * [new branch] gh/mikaylagawarecki/341/orig -> origin/gh/mikaylagawarecki/341/orig 2025-12-04T09:21:04.2753558Z * [new branch] gh/mikaylagawarecki/342/base -> origin/gh/mikaylagawarecki/342/base 2025-12-04T09:21:04.2754785Z * [new branch] gh/mikaylagawarecki/342/head -> origin/gh/mikaylagawarecki/342/head 2025-12-04T09:21:04.2756093Z * [new branch] gh/mikaylagawarecki/342/orig -> origin/gh/mikaylagawarecki/342/orig 2025-12-04T09:21:04.2757767Z * [new branch] gh/mikaylagawarecki/345/base -> origin/gh/mikaylagawarecki/345/base 2025-12-04T09:21:04.2758931Z * [new branch] gh/mikaylagawarecki/345/head -> origin/gh/mikaylagawarecki/345/head 2025-12-04T09:21:04.2760275Z * [new branch] gh/mikaylagawarecki/345/orig -> origin/gh/mikaylagawarecki/345/orig 2025-12-04T09:21:04.2761990Z * [new branch] gh/mikaylagawarecki/346/base -> origin/gh/mikaylagawarecki/346/base 2025-12-04T09:21:04.2763158Z * [new branch] gh/mikaylagawarecki/346/head -> origin/gh/mikaylagawarecki/346/head 2025-12-04T09:21:04.2764408Z * [new branch] gh/mikaylagawarecki/346/orig -> origin/gh/mikaylagawarecki/346/orig 2025-12-04T09:21:04.2766238Z * [new branch] gh/mikaylagawarecki/347/base -> origin/gh/mikaylagawarecki/347/base 2025-12-04T09:21:04.2767399Z * [new branch] gh/mikaylagawarecki/347/head -> origin/gh/mikaylagawarecki/347/head 2025-12-04T09:21:04.2768596Z * [new branch] gh/mikaylagawarecki/347/orig -> origin/gh/mikaylagawarecki/347/orig 2025-12-04T09:21:04.2770244Z * [new branch] gh/mikaylagawarecki/350/base -> origin/gh/mikaylagawarecki/350/base 2025-12-04T09:21:04.2771463Z * [new branch] gh/mikaylagawarecki/350/head -> origin/gh/mikaylagawarecki/350/head 2025-12-04T09:21:04.2772629Z * [new branch] gh/mikaylagawarecki/350/orig -> origin/gh/mikaylagawarecki/350/orig 2025-12-04T09:21:04.2774809Z * [new branch] gh/mikaylagawarecki/351/base -> origin/gh/mikaylagawarecki/351/base 2025-12-04T09:21:04.2781036Z * [new branch] gh/mikaylagawarecki/351/head -> origin/gh/mikaylagawarecki/351/head 2025-12-04T09:21:04.2781532Z * [new branch] gh/mikaylagawarecki/351/orig -> origin/gh/mikaylagawarecki/351/orig 2025-12-04T09:21:04.2781876Z * [new branch] gh/mikaylagawarecki/352/base -> origin/gh/mikaylagawarecki/352/base 2025-12-04T09:21:04.2782202Z * [new branch] gh/mikaylagawarecki/352/head -> origin/gh/mikaylagawarecki/352/head 2025-12-04T09:21:04.2782835Z * [new branch] gh/mikaylagawarecki/352/orig -> origin/gh/mikaylagawarecki/352/orig 2025-12-04T09:21:04.2784827Z * [new branch] gh/mikaylagawarecki/353/base -> origin/gh/mikaylagawarecki/353/base 2025-12-04T09:21:04.2786279Z * [new branch] gh/mikaylagawarecki/353/head -> origin/gh/mikaylagawarecki/353/head 2025-12-04T09:21:04.2787742Z * [new branch] gh/mikaylagawarecki/353/orig -> origin/gh/mikaylagawarecki/353/orig 2025-12-04T09:21:04.2789201Z * [new branch] gh/mikaylagawarecki/354/base -> origin/gh/mikaylagawarecki/354/base 2025-12-04T09:21:04.2790470Z * [new branch] gh/mikaylagawarecki/354/head -> origin/gh/mikaylagawarecki/354/head 2025-12-04T09:21:04.2791708Z * [new branch] gh/mikaylagawarecki/354/orig -> origin/gh/mikaylagawarecki/354/orig 2025-12-04T09:21:04.2793769Z * [new branch] gh/mikaylagawarecki/356/base -> origin/gh/mikaylagawarecki/356/base 2025-12-04T09:21:04.2795068Z * [new branch] gh/mikaylagawarecki/356/head -> origin/gh/mikaylagawarecki/356/head 2025-12-04T09:21:04.2796342Z * [new branch] gh/mikaylagawarecki/356/orig -> origin/gh/mikaylagawarecki/356/orig 2025-12-04T09:21:04.2798302Z * [new branch] gh/mikaylagawarecki/357/base -> origin/gh/mikaylagawarecki/357/base 2025-12-04T09:21:04.2799494Z * [new branch] gh/mikaylagawarecki/357/head -> origin/gh/mikaylagawarecki/357/head 2025-12-04T09:21:04.2800703Z * [new branch] gh/mikaylagawarecki/357/orig -> origin/gh/mikaylagawarecki/357/orig 2025-12-04T09:21:04.2803375Z * [new branch] gh/mikaylagawarecki/359/base -> origin/gh/mikaylagawarecki/359/base 2025-12-04T09:21:04.2804736Z * [new branch] gh/mikaylagawarecki/359/head -> origin/gh/mikaylagawarecki/359/head 2025-12-04T09:21:04.2805997Z * [new branch] gh/mikaylagawarecki/359/orig -> origin/gh/mikaylagawarecki/359/orig 2025-12-04T09:21:04.2807699Z * [new branch] gh/mikaylagawarecki/360/base -> origin/gh/mikaylagawarecki/360/base 2025-12-04T09:21:04.2809098Z * [new branch] gh/mikaylagawarecki/360/head -> origin/gh/mikaylagawarecki/360/head 2025-12-04T09:21:04.2810330Z * [new branch] gh/mikaylagawarecki/360/orig -> origin/gh/mikaylagawarecki/360/orig 2025-12-04T09:21:04.2811968Z * [new branch] gh/mikaylagawarecki/361/base -> origin/gh/mikaylagawarecki/361/base 2025-12-04T09:21:04.2813265Z * [new branch] gh/mikaylagawarecki/361/head -> origin/gh/mikaylagawarecki/361/head 2025-12-04T09:21:04.2814609Z * [new branch] gh/mikaylagawarecki/361/orig -> origin/gh/mikaylagawarecki/361/orig 2025-12-04T09:21:04.2816310Z * [new branch] gh/mikaylagawarecki/362/base -> origin/gh/mikaylagawarecki/362/base 2025-12-04T09:21:04.2817730Z * [new branch] gh/mikaylagawarecki/362/head -> origin/gh/mikaylagawarecki/362/head 2025-12-04T09:21:04.2818954Z * [new branch] gh/mikaylagawarecki/362/orig -> origin/gh/mikaylagawarecki/362/orig 2025-12-04T09:21:04.2821069Z * [new branch] gh/mikaylagawarecki/363/base -> origin/gh/mikaylagawarecki/363/base 2025-12-04T09:21:04.2822552Z * [new branch] gh/mikaylagawarecki/363/head -> origin/gh/mikaylagawarecki/363/head 2025-12-04T09:21:04.2823726Z * [new branch] gh/mikaylagawarecki/363/orig -> origin/gh/mikaylagawarecki/363/orig 2025-12-04T09:21:04.2825836Z * [new branch] gh/mikaylagawarecki/364/base -> origin/gh/mikaylagawarecki/364/base 2025-12-04T09:21:04.2827270Z * [new branch] gh/mikaylagawarecki/364/head -> origin/gh/mikaylagawarecki/364/head 2025-12-04T09:21:04.2828711Z * [new branch] gh/mikaylagawarecki/364/orig -> origin/gh/mikaylagawarecki/364/orig 2025-12-04T09:21:04.2830618Z * [new branch] gh/mikaylagawarecki/365/base -> origin/gh/mikaylagawarecki/365/base 2025-12-04T09:21:04.2832011Z * [new branch] gh/mikaylagawarecki/365/head -> origin/gh/mikaylagawarecki/365/head 2025-12-04T09:21:04.2833361Z * [new branch] gh/mikaylagawarecki/365/orig -> origin/gh/mikaylagawarecki/365/orig 2025-12-04T09:21:04.2835077Z * [new branch] gh/mikaylagawarecki/366/base -> origin/gh/mikaylagawarecki/366/base 2025-12-04T09:21:04.2836258Z * [new branch] gh/mikaylagawarecki/366/head -> origin/gh/mikaylagawarecki/366/head 2025-12-04T09:21:04.2842449Z * [new branch] gh/mikaylagawarecki/366/orig -> origin/gh/mikaylagawarecki/366/orig 2025-12-04T09:21:04.2842774Z * [new branch] gh/mikaylagawarecki/367/base -> origin/gh/mikaylagawarecki/367/base 2025-12-04T09:21:04.2843120Z * [new branch] gh/mikaylagawarecki/367/head -> origin/gh/mikaylagawarecki/367/head 2025-12-04T09:21:04.2843450Z * [new branch] gh/mikaylagawarecki/367/orig -> origin/gh/mikaylagawarecki/367/orig 2025-12-04T09:21:04.2844002Z * [new branch] gh/mikaylagawarecki/368/base -> origin/gh/mikaylagawarecki/368/base 2025-12-04T09:21:04.2845280Z * [new branch] gh/mikaylagawarecki/368/head -> origin/gh/mikaylagawarecki/368/head 2025-12-04T09:21:04.2846519Z * [new branch] gh/mikaylagawarecki/368/orig -> origin/gh/mikaylagawarecki/368/orig 2025-12-04T09:21:04.2848226Z * [new branch] gh/mikaylagawarecki/369/base -> origin/gh/mikaylagawarecki/369/base 2025-12-04T09:21:04.2849611Z * [new branch] gh/mikaylagawarecki/369/head -> origin/gh/mikaylagawarecki/369/head 2025-12-04T09:21:04.2850836Z * [new branch] gh/mikaylagawarecki/369/orig -> origin/gh/mikaylagawarecki/369/orig 2025-12-04T09:21:04.2852585Z * [new branch] gh/mikaylagawarecki/370/base -> origin/gh/mikaylagawarecki/370/base 2025-12-04T09:21:04.2853887Z * [new branch] gh/mikaylagawarecki/370/head -> origin/gh/mikaylagawarecki/370/head 2025-12-04T09:21:04.2855020Z * [new branch] gh/mikaylagawarecki/370/orig -> origin/gh/mikaylagawarecki/370/orig 2025-12-04T09:21:04.2856788Z * [new branch] gh/mikaylagawarecki/371/base -> origin/gh/mikaylagawarecki/371/base 2025-12-04T09:21:04.2858006Z * [new branch] gh/mikaylagawarecki/371/head -> origin/gh/mikaylagawarecki/371/head 2025-12-04T09:21:04.2859204Z * [new branch] gh/mikaylagawarecki/371/orig -> origin/gh/mikaylagawarecki/371/orig 2025-12-04T09:21:04.2861281Z * [new branch] gh/mikaylagawarecki/372/base -> origin/gh/mikaylagawarecki/372/base 2025-12-04T09:21:04.2862477Z * [new branch] gh/mikaylagawarecki/372/head -> origin/gh/mikaylagawarecki/372/head 2025-12-04T09:21:04.2863718Z * [new branch] gh/mikaylagawarecki/372/orig -> origin/gh/mikaylagawarecki/372/orig 2025-12-04T09:21:04.2865256Z * [new branch] gh/mikaylagawarecki/373/base -> origin/gh/mikaylagawarecki/373/base 2025-12-04T09:21:04.2866494Z * [new branch] gh/mikaylagawarecki/373/head -> origin/gh/mikaylagawarecki/373/head 2025-12-04T09:21:04.2868070Z * [new branch] gh/mikaylagawarecki/373/orig -> origin/gh/mikaylagawarecki/373/orig 2025-12-04T09:21:04.2869763Z * [new branch] gh/mikaylagawarecki/374/base -> origin/gh/mikaylagawarecki/374/base 2025-12-04T09:21:04.2871048Z * [new branch] gh/mikaylagawarecki/374/head -> origin/gh/mikaylagawarecki/374/head 2025-12-04T09:21:04.2872255Z * [new branch] gh/mikaylagawarecki/374/orig -> origin/gh/mikaylagawarecki/374/orig 2025-12-04T09:21:04.2874563Z * [new branch] gh/mikaylagawarecki/375/base -> origin/gh/mikaylagawarecki/375/base 2025-12-04T09:21:04.2875863Z * [new branch] gh/mikaylagawarecki/375/head -> origin/gh/mikaylagawarecki/375/head 2025-12-04T09:21:04.2877181Z * [new branch] gh/mikaylagawarecki/375/orig -> origin/gh/mikaylagawarecki/375/orig 2025-12-04T09:21:04.2878989Z * [new branch] gh/mikaylagawarecki/376/base -> origin/gh/mikaylagawarecki/376/base 2025-12-04T09:21:04.2880333Z * [new branch] gh/mikaylagawarecki/376/head -> origin/gh/mikaylagawarecki/376/head 2025-12-04T09:21:04.2881481Z * [new branch] gh/mikaylagawarecki/376/orig -> origin/gh/mikaylagawarecki/376/orig 2025-12-04T09:21:04.2883168Z * [new branch] gh/mikaylagawarecki/377/base -> origin/gh/mikaylagawarecki/377/base 2025-12-04T09:21:04.2884469Z * [new branch] gh/mikaylagawarecki/377/head -> origin/gh/mikaylagawarecki/377/head 2025-12-04T09:21:04.2885724Z * [new branch] gh/mikaylagawarecki/377/orig -> origin/gh/mikaylagawarecki/377/orig 2025-12-04T09:21:04.2887376Z * [new branch] gh/mikaylagawarecki/378/base -> origin/gh/mikaylagawarecki/378/base 2025-12-04T09:21:04.2888629Z * [new branch] gh/mikaylagawarecki/378/head -> origin/gh/mikaylagawarecki/378/head 2025-12-04T09:21:04.2889851Z * [new branch] gh/mikaylagawarecki/378/orig -> origin/gh/mikaylagawarecki/378/orig 2025-12-04T09:21:04.2891601Z * [new branch] gh/mikaylagawarecki/379/base -> origin/gh/mikaylagawarecki/379/base 2025-12-04T09:21:04.2892784Z * [new branch] gh/mikaylagawarecki/379/head -> origin/gh/mikaylagawarecki/379/head 2025-12-04T09:21:04.2893989Z * [new branch] gh/mikaylagawarecki/379/orig -> origin/gh/mikaylagawarecki/379/orig 2025-12-04T09:21:04.2895460Z * [new branch] gh/mikaylagawarecki/380/base -> origin/gh/mikaylagawarecki/380/base 2025-12-04T09:21:04.2896652Z * [new branch] gh/mikaylagawarecki/380/head -> origin/gh/mikaylagawarecki/380/head 2025-12-04T09:21:04.2897841Z * [new branch] gh/mikaylagawarecki/380/orig -> origin/gh/mikaylagawarecki/380/orig 2025-12-04T09:21:04.2899318Z * [new branch] gh/mikaylagawarecki/381/base -> origin/gh/mikaylagawarecki/381/base 2025-12-04T09:21:04.2907323Z * [new branch] gh/mikaylagawarecki/381/head -> origin/gh/mikaylagawarecki/381/head 2025-12-04T09:21:04.2908639Z * [new branch] gh/mikaylagawarecki/381/orig -> origin/gh/mikaylagawarecki/381/orig 2025-12-04T09:21:04.2910878Z * [new branch] gh/mikaylagawarecki/382/base -> origin/gh/mikaylagawarecki/382/base 2025-12-04T09:21:04.2912123Z * [new branch] gh/mikaylagawarecki/382/head -> origin/gh/mikaylagawarecki/382/head 2025-12-04T09:21:04.2913399Z * [new branch] gh/mikaylagawarecki/382/orig -> origin/gh/mikaylagawarecki/382/orig 2025-12-04T09:21:04.2915178Z * [new branch] gh/mikaylagawarecki/383/base -> origin/gh/mikaylagawarecki/383/base 2025-12-04T09:21:04.2916566Z * [new branch] gh/mikaylagawarecki/383/head -> origin/gh/mikaylagawarecki/383/head 2025-12-04T09:21:04.2917777Z * [new branch] gh/mikaylagawarecki/383/orig -> origin/gh/mikaylagawarecki/383/orig 2025-12-04T09:21:04.2919561Z * [new branch] gh/mikaylagawarecki/384/base -> origin/gh/mikaylagawarecki/384/base 2025-12-04T09:21:04.2920832Z * [new branch] gh/mikaylagawarecki/384/head -> origin/gh/mikaylagawarecki/384/head 2025-12-04T09:21:04.2922049Z * [new branch] gh/mikaylagawarecki/384/orig -> origin/gh/mikaylagawarecki/384/orig 2025-12-04T09:21:04.2923679Z * [new branch] gh/mikaylagawarecki/385/base -> origin/gh/mikaylagawarecki/385/base 2025-12-04T09:21:04.2924919Z * [new branch] gh/mikaylagawarecki/385/head -> origin/gh/mikaylagawarecki/385/head 2025-12-04T09:21:04.2926125Z * [new branch] gh/mikaylagawarecki/385/orig -> origin/gh/mikaylagawarecki/385/orig 2025-12-04T09:21:04.2928103Z * [new branch] gh/mikaylagawarecki/386/base -> origin/gh/mikaylagawarecki/386/base 2025-12-04T09:21:04.2929260Z * [new branch] gh/mikaylagawarecki/386/head -> origin/gh/mikaylagawarecki/386/head 2025-12-04T09:21:04.2930475Z * [new branch] gh/mikaylagawarecki/386/orig -> origin/gh/mikaylagawarecki/386/orig 2025-12-04T09:21:04.2932650Z * [new branch] gh/mikaylagawarecki/387/base -> origin/gh/mikaylagawarecki/387/base 2025-12-04T09:21:04.2933857Z * [new branch] gh/mikaylagawarecki/387/head -> origin/gh/mikaylagawarecki/387/head 2025-12-04T09:21:04.2935033Z * [new branch] gh/mikaylagawarecki/387/orig -> origin/gh/mikaylagawarecki/387/orig 2025-12-04T09:21:04.2936552Z * [new branch] gh/mikaylagawarecki/388/base -> origin/gh/mikaylagawarecki/388/base 2025-12-04T09:21:04.2937756Z * [new branch] gh/mikaylagawarecki/388/head -> origin/gh/mikaylagawarecki/388/head 2025-12-04T09:21:04.2938946Z * [new branch] gh/mikaylagawarecki/388/orig -> origin/gh/mikaylagawarecki/388/orig 2025-12-04T09:21:04.2940678Z * [new branch] gh/mikaylagawarecki/389/base -> origin/gh/mikaylagawarecki/389/base 2025-12-04T09:21:04.2941870Z * [new branch] gh/mikaylagawarecki/389/head -> origin/gh/mikaylagawarecki/389/head 2025-12-04T09:21:04.2943087Z * [new branch] gh/mikaylagawarecki/389/orig -> origin/gh/mikaylagawarecki/389/orig 2025-12-04T09:21:04.2944870Z * [new branch] gh/mikaylagawarecki/390/base -> origin/gh/mikaylagawarecki/390/base 2025-12-04T09:21:04.2946045Z * [new branch] gh/mikaylagawarecki/390/head -> origin/gh/mikaylagawarecki/390/head 2025-12-04T09:21:04.2947691Z * [new branch] gh/mikaylagawarecki/390/orig -> origin/gh/mikaylagawarecki/390/orig 2025-12-04T09:21:04.2949535Z * [new branch] gh/mikaylagawarecki/391/base -> origin/gh/mikaylagawarecki/391/base 2025-12-04T09:21:04.2950897Z * [new branch] gh/mikaylagawarecki/391/head -> origin/gh/mikaylagawarecki/391/head 2025-12-04T09:21:04.2952131Z * [new branch] gh/mikaylagawarecki/391/orig -> origin/gh/mikaylagawarecki/391/orig 2025-12-04T09:21:04.2953980Z * [new branch] gh/mikaylagawarecki/392/base -> origin/gh/mikaylagawarecki/392/base 2025-12-04T09:21:04.2955242Z * [new branch] gh/mikaylagawarecki/392/head -> origin/gh/mikaylagawarecki/392/head 2025-12-04T09:21:04.2956444Z * [new branch] gh/mikaylagawarecki/392/orig -> origin/gh/mikaylagawarecki/392/orig 2025-12-04T09:21:04.2958444Z * [new branch] gh/mlazos/41/base -> origin/gh/mlazos/41/base 2025-12-04T09:21:04.2959763Z * [new branch] gh/mlazos/41/head -> origin/gh/mlazos/41/head 2025-12-04T09:21:04.2961094Z * [new branch] gh/mlazos/41/orig -> origin/gh/mlazos/41/orig 2025-12-04T09:21:04.2967507Z * [new branch] gh/mlazos/42/base -> origin/gh/mlazos/42/base 2025-12-04T09:21:04.2967777Z * [new branch] gh/mlazos/42/head -> origin/gh/mlazos/42/head 2025-12-04T09:21:04.2968024Z * [new branch] gh/mlazos/42/orig -> origin/gh/mlazos/42/orig 2025-12-04T09:21:04.2968268Z * [new branch] gh/mlazos/43/base -> origin/gh/mlazos/43/base 2025-12-04T09:21:04.2968893Z * [new branch] gh/mlazos/43/head -> origin/gh/mlazos/43/head 2025-12-04T09:21:04.2969793Z * [new branch] gh/mlazos/43/orig -> origin/gh/mlazos/43/orig 2025-12-04T09:21:04.2971487Z * [new branch] gh/mlazos/44/base -> origin/gh/mlazos/44/base 2025-12-04T09:21:04.2972412Z * [new branch] gh/mlazos/44/head -> origin/gh/mlazos/44/head 2025-12-04T09:21:04.2973664Z * [new branch] gh/mlazos/44/orig -> origin/gh/mlazos/44/orig 2025-12-04T09:21:04.2975231Z * [new branch] gh/mlazos/47/base -> origin/gh/mlazos/47/base 2025-12-04T09:21:04.2976448Z * [new branch] gh/mlazos/47/head -> origin/gh/mlazos/47/head 2025-12-04T09:21:04.2977817Z * [new branch] gh/mlazos/47/orig -> origin/gh/mlazos/47/orig 2025-12-04T09:21:04.2979447Z * [new branch] gh/mlazos/48/base -> origin/gh/mlazos/48/base 2025-12-04T09:21:04.2980712Z * [new branch] gh/mlazos/48/head -> origin/gh/mlazos/48/head 2025-12-04T09:21:04.2981794Z * [new branch] gh/mlazos/48/orig -> origin/gh/mlazos/48/orig 2025-12-04T09:21:04.2983359Z * [new branch] gh/mlazos/49/base -> origin/gh/mlazos/49/base 2025-12-04T09:21:04.2984487Z * [new branch] gh/mlazos/49/head -> origin/gh/mlazos/49/head 2025-12-04T09:21:04.2985686Z * [new branch] gh/mlazos/49/orig -> origin/gh/mlazos/49/orig 2025-12-04T09:21:04.2987682Z * [new branch] gh/mlazos/50/base -> origin/gh/mlazos/50/base 2025-12-04T09:21:04.2988841Z * [new branch] gh/mlazos/50/head -> origin/gh/mlazos/50/head 2025-12-04T09:21:04.2990046Z * [new branch] gh/mlazos/50/orig -> origin/gh/mlazos/50/orig 2025-12-04T09:21:04.2991587Z * [new branch] gh/mlazos/51/base -> origin/gh/mlazos/51/base 2025-12-04T09:21:04.2992766Z * [new branch] gh/mlazos/51/head -> origin/gh/mlazos/51/head 2025-12-04T09:21:04.2994056Z * [new branch] gh/mlazos/51/orig -> origin/gh/mlazos/51/orig 2025-12-04T09:21:04.2995685Z * [new branch] gh/mlazos/52/base -> origin/gh/mlazos/52/base 2025-12-04T09:21:04.2996879Z * [new branch] gh/mlazos/52/head -> origin/gh/mlazos/52/head 2025-12-04T09:21:04.2998092Z * [new branch] gh/mlazos/52/orig -> origin/gh/mlazos/52/orig 2025-12-04T09:21:04.2999837Z * [new branch] gh/mlazos/53/base -> origin/gh/mlazos/53/base 2025-12-04T09:21:04.3001743Z * [new branch] gh/mlazos/53/head -> origin/gh/mlazos/53/head 2025-12-04T09:21:04.3003011Z * [new branch] gh/mlazos/53/orig -> origin/gh/mlazos/53/orig 2025-12-04T09:21:04.3005079Z * [new branch] gh/mlazos/54/base -> origin/gh/mlazos/54/base 2025-12-04T09:21:04.3006315Z * [new branch] gh/mlazos/54/head -> origin/gh/mlazos/54/head 2025-12-04T09:21:04.3007526Z * [new branch] gh/mlazos/54/orig -> origin/gh/mlazos/54/orig 2025-12-04T09:21:04.3009354Z * [new branch] gh/mlazos/55/base -> origin/gh/mlazos/55/base 2025-12-04T09:21:04.3010658Z * [new branch] gh/mlazos/55/head -> origin/gh/mlazos/55/head 2025-12-04T09:21:04.3011965Z * [new branch] gh/mlazos/55/orig -> origin/gh/mlazos/55/orig 2025-12-04T09:21:04.3013600Z * [new branch] gh/mlazos/56/base -> origin/gh/mlazos/56/base 2025-12-04T09:21:04.3014879Z * [new branch] gh/mlazos/56/head -> origin/gh/mlazos/56/head 2025-12-04T09:21:04.3016044Z * [new branch] gh/mlazos/56/orig -> origin/gh/mlazos/56/orig 2025-12-04T09:21:04.3017619Z * [new branch] gh/mlazos/57/base -> origin/gh/mlazos/57/base 2025-12-04T09:21:04.3018777Z * [new branch] gh/mlazos/57/head -> origin/gh/mlazos/57/head 2025-12-04T09:21:04.3019946Z * [new branch] gh/mlazos/57/orig -> origin/gh/mlazos/57/orig 2025-12-04T09:21:04.3021559Z * [new branch] gh/mlazos/58/base -> origin/gh/mlazos/58/base 2025-12-04T09:21:04.3022757Z * [new branch] gh/mlazos/58/head -> origin/gh/mlazos/58/head 2025-12-04T09:21:04.3023967Z * [new branch] gh/mlazos/58/orig -> origin/gh/mlazos/58/orig 2025-12-04T09:21:04.3030001Z * [new branch] gh/mlazos/59/base -> origin/gh/mlazos/59/base 2025-12-04T09:21:04.3030412Z * [new branch] gh/mlazos/59/head -> origin/gh/mlazos/59/head 2025-12-04T09:21:04.3030654Z * [new branch] gh/mlazos/59/orig -> origin/gh/mlazos/59/orig 2025-12-04T09:21:04.3030965Z * [new branch] gh/mlazos/60/base -> origin/gh/mlazos/60/base 2025-12-04T09:21:04.3032190Z * [new branch] gh/mlazos/60/head -> origin/gh/mlazos/60/head 2025-12-04T09:21:04.3033325Z * [new branch] gh/mlazos/60/orig -> origin/gh/mlazos/60/orig 2025-12-04T09:21:04.3035413Z * [new branch] gh/mlazos/61/base -> origin/gh/mlazos/61/base 2025-12-04T09:21:04.3036764Z * [new branch] gh/mlazos/61/head -> origin/gh/mlazos/61/head 2025-12-04T09:21:04.3037946Z * [new branch] gh/mlazos/61/orig -> origin/gh/mlazos/61/orig 2025-12-04T09:21:04.3039657Z * [new branch] gh/mlazos/62/base -> origin/gh/mlazos/62/base 2025-12-04T09:21:04.3040825Z * [new branch] gh/mlazos/62/head -> origin/gh/mlazos/62/head 2025-12-04T09:21:04.3042077Z * [new branch] gh/mlazos/62/orig -> origin/gh/mlazos/62/orig 2025-12-04T09:21:04.3044143Z * [new branch] gh/mlazos/63/base -> origin/gh/mlazos/63/base 2025-12-04T09:21:04.3045422Z * [new branch] gh/mlazos/63/head -> origin/gh/mlazos/63/head 2025-12-04T09:21:04.3046773Z * [new branch] gh/mlazos/63/orig -> origin/gh/mlazos/63/orig 2025-12-04T09:21:04.3048362Z * [new branch] gh/mlazos/64/base -> origin/gh/mlazos/64/base 2025-12-04T09:21:04.3049552Z * [new branch] gh/mlazos/64/head -> origin/gh/mlazos/64/head 2025-12-04T09:21:04.3051219Z * [new branch] gh/mlazos/64/orig -> origin/gh/mlazos/64/orig 2025-12-04T09:21:04.3052821Z * [new branch] gh/mlazos/65/base -> origin/gh/mlazos/65/base 2025-12-04T09:21:04.3054040Z * [new branch] gh/mlazos/65/head -> origin/gh/mlazos/65/head 2025-12-04T09:21:04.3055254Z * [new branch] gh/mlazos/65/orig -> origin/gh/mlazos/65/orig 2025-12-04T09:21:04.3056836Z * [new branch] gh/mlazos/66/base -> origin/gh/mlazos/66/base 2025-12-04T09:21:04.3058082Z * [new branch] gh/mlazos/66/head -> origin/gh/mlazos/66/head 2025-12-04T09:21:04.3059286Z * [new branch] gh/mlazos/66/orig -> origin/gh/mlazos/66/orig 2025-12-04T09:21:04.3060880Z * [new branch] gh/mlazos/67/base -> origin/gh/mlazos/67/base 2025-12-04T09:21:04.3062069Z * [new branch] gh/mlazos/67/head -> origin/gh/mlazos/67/head 2025-12-04T09:21:04.3063381Z * [new branch] gh/mlazos/67/orig -> origin/gh/mlazos/67/orig 2025-12-04T09:21:04.3065037Z * [new branch] gh/mlazos/68/base -> origin/gh/mlazos/68/base 2025-12-04T09:21:04.3066207Z * [new branch] gh/mlazos/68/head -> origin/gh/mlazos/68/head 2025-12-04T09:21:04.3067804Z * [new branch] gh/mlazos/68/orig -> origin/gh/mlazos/68/orig 2025-12-04T09:21:04.3069481Z * [new branch] gh/mlazos/69/base -> origin/gh/mlazos/69/base 2025-12-04T09:21:04.3070794Z * [new branch] gh/mlazos/69/head -> origin/gh/mlazos/69/head 2025-12-04T09:21:04.3072011Z * [new branch] gh/mlazos/69/orig -> origin/gh/mlazos/69/orig 2025-12-04T09:21:04.3073651Z * [new branch] gh/mlazos/70/base -> origin/gh/mlazos/70/base 2025-12-04T09:21:04.3074868Z * [new branch] gh/mlazos/70/head -> origin/gh/mlazos/70/head 2025-12-04T09:21:04.3076111Z * [new branch] gh/mlazos/70/orig -> origin/gh/mlazos/70/orig 2025-12-04T09:21:04.3077814Z * [new branch] gh/mlazos/71/base -> origin/gh/mlazos/71/base 2025-12-04T09:21:04.3079150Z * [new branch] gh/mlazos/71/head -> origin/gh/mlazos/71/head 2025-12-04T09:21:04.3080457Z * [new branch] gh/mlazos/71/orig -> origin/gh/mlazos/71/orig 2025-12-04T09:21:04.3081987Z * [new branch] gh/mlazos/72/base -> origin/gh/mlazos/72/base 2025-12-04T09:21:04.3083260Z * [new branch] gh/mlazos/72/head -> origin/gh/mlazos/72/head 2025-12-04T09:21:04.3084409Z * [new branch] gh/mlazos/72/orig -> origin/gh/mlazos/72/orig 2025-12-04T09:21:04.3086079Z * [new branch] gh/mlazos/73/base -> origin/gh/mlazos/73/base 2025-12-04T09:21:04.3092408Z * [new branch] gh/mlazos/73/head -> origin/gh/mlazos/73/head 2025-12-04T09:21:04.3094332Z * [new branch] gh/mlazos/73/orig -> origin/gh/mlazos/73/orig 2025-12-04T09:21:04.3096146Z * [new branch] gh/mrmiywj/1/base -> origin/gh/mrmiywj/1/base 2025-12-04T09:21:04.3097440Z * [new branch] gh/mrmiywj/1/head -> origin/gh/mrmiywj/1/head 2025-12-04T09:21:04.3099745Z * [new branch] gh/muchulee8/73/base -> origin/gh/muchulee8/73/base 2025-12-04T09:21:04.3101543Z * [new branch] gh/muchulee8/73/head -> origin/gh/muchulee8/73/head 2025-12-04T09:21:04.3103063Z * [new branch] gh/muchulee8/73/orig -> origin/gh/muchulee8/73/orig 2025-12-04T09:21:04.3105120Z * [new branch] gh/naveenthangudu/1/base -> origin/gh/naveenthangudu/1/base 2025-12-04T09:21:04.3106350Z * [new branch] gh/naveenthangudu/1/head -> origin/gh/naveenthangudu/1/head 2025-12-04T09:21:04.3107914Z * [new branch] gh/naveenthangudu/1/orig -> origin/gh/naveenthangudu/1/orig 2025-12-04T09:21:04.3110001Z * [new branch] gh/naveenthangudu/2/base -> origin/gh/naveenthangudu/2/base 2025-12-04T09:21:04.3111306Z * [new branch] gh/naveenthangudu/2/head -> origin/gh/naveenthangudu/2/head 2025-12-04T09:21:04.3112555Z * [new branch] gh/naveenthangudu/2/orig -> origin/gh/naveenthangudu/2/orig 2025-12-04T09:21:04.3114052Z * [new branch] gh/naveenthangudu/3/base -> origin/gh/naveenthangudu/3/base 2025-12-04T09:21:04.3115292Z * [new branch] gh/naveenthangudu/3/head -> origin/gh/naveenthangudu/3/head 2025-12-04T09:21:04.3116673Z * [new branch] gh/naveenthangudu/3/orig -> origin/gh/naveenthangudu/3/orig 2025-12-04T09:21:04.3118582Z * [new branch] gh/naveenthangudu/4/base -> origin/gh/naveenthangudu/4/base 2025-12-04T09:21:04.3119790Z * [new branch] gh/naveenthangudu/4/head -> origin/gh/naveenthangudu/4/head 2025-12-04T09:21:04.3121082Z * [new branch] gh/naveenthangudu/4/orig -> origin/gh/naveenthangudu/4/orig 2025-12-04T09:21:04.3122964Z * [new branch] gh/naveenthangudu/5/base -> origin/gh/naveenthangudu/5/base 2025-12-04T09:21:04.3124172Z * [new branch] gh/naveenthangudu/5/head -> origin/gh/naveenthangudu/5/head 2025-12-04T09:21:04.3125585Z * [new branch] gh/naveenthangudu/5/orig -> origin/gh/naveenthangudu/5/orig 2025-12-04T09:21:04.3127156Z * [new branch] gh/naveenthangudu/6/base -> origin/gh/naveenthangudu/6/base 2025-12-04T09:21:04.3128380Z * [new branch] gh/naveenthangudu/6/head -> origin/gh/naveenthangudu/6/head 2025-12-04T09:21:04.3129532Z * [new branch] gh/naveenthangudu/6/orig -> origin/gh/naveenthangudu/6/orig 2025-12-04T09:21:04.3131097Z * [new branch] gh/naveenthangudu/7/base -> origin/gh/naveenthangudu/7/base 2025-12-04T09:21:04.3132304Z * [new branch] gh/naveenthangudu/7/head -> origin/gh/naveenthangudu/7/head 2025-12-04T09:21:04.3133426Z * [new branch] gh/naveenthangudu/7/orig -> origin/gh/naveenthangudu/7/orig 2025-12-04T09:21:04.3135773Z * [new branch] gh/naveenthangudu/8/base -> origin/gh/naveenthangudu/8/base 2025-12-04T09:21:04.3136479Z * [new branch] gh/naveenthangudu/8/head -> origin/gh/naveenthangudu/8/head 2025-12-04T09:21:04.3137744Z * [new branch] gh/naveenthangudu/8/orig -> origin/gh/naveenthangudu/8/orig 2025-12-04T09:21:04.3139377Z * [new branch] gh/naveenthangudu/9/base -> origin/gh/naveenthangudu/9/base 2025-12-04T09:21:04.3140469Z * [new branch] gh/naveenthangudu/9/head -> origin/gh/naveenthangudu/9/head 2025-12-04T09:21:04.3141736Z * [new branch] gh/naveenthangudu/9/orig -> origin/gh/naveenthangudu/9/orig 2025-12-04T09:21:04.3143617Z * [new branch] gh/nikitaved/1/base -> origin/gh/nikitaved/1/base 2025-12-04T09:21:04.3144831Z * [new branch] gh/nikitaved/1/head -> origin/gh/nikitaved/1/head 2025-12-04T09:21:04.3146004Z * [new branch] gh/nikitaved/1/orig -> origin/gh/nikitaved/1/orig 2025-12-04T09:21:04.3148082Z * [new branch] gh/nikitaved/10/base -> origin/gh/nikitaved/10/base 2025-12-04T09:21:04.3149294Z * [new branch] gh/nikitaved/10/head -> origin/gh/nikitaved/10/head 2025-12-04T09:21:04.3155428Z * [new branch] gh/nikitaved/10/orig -> origin/gh/nikitaved/10/orig 2025-12-04T09:21:04.3157191Z * [new branch] gh/nikitaved/11/base -> origin/gh/nikitaved/11/base 2025-12-04T09:21:04.3158524Z * [new branch] gh/nikitaved/11/head -> origin/gh/nikitaved/11/head 2025-12-04T09:21:04.3159782Z * [new branch] gh/nikitaved/11/orig -> origin/gh/nikitaved/11/orig 2025-12-04T09:21:04.3161368Z * [new branch] gh/nikitaved/12/base -> origin/gh/nikitaved/12/base 2025-12-04T09:21:04.3162698Z * [new branch] gh/nikitaved/12/head -> origin/gh/nikitaved/12/head 2025-12-04T09:21:04.3163894Z * [new branch] gh/nikitaved/12/orig -> origin/gh/nikitaved/12/orig 2025-12-04T09:21:04.3166075Z * [new branch] gh/nikitaved/13/base -> origin/gh/nikitaved/13/base 2025-12-04T09:21:04.3167272Z * [new branch] gh/nikitaved/13/head -> origin/gh/nikitaved/13/head 2025-12-04T09:21:04.3168464Z * [new branch] gh/nikitaved/13/orig -> origin/gh/nikitaved/13/orig 2025-12-04T09:21:04.3170089Z * [new branch] gh/nikitaved/14/base -> origin/gh/nikitaved/14/base 2025-12-04T09:21:04.3171329Z * [new branch] gh/nikitaved/14/head -> origin/gh/nikitaved/14/head 2025-12-04T09:21:04.3172521Z * [new branch] gh/nikitaved/14/orig -> origin/gh/nikitaved/14/orig 2025-12-04T09:21:04.3174094Z * [new branch] gh/nikitaved/15/base -> origin/gh/nikitaved/15/base 2025-12-04T09:21:04.3175297Z * [new branch] gh/nikitaved/15/head -> origin/gh/nikitaved/15/head 2025-12-04T09:21:04.3176465Z * [new branch] gh/nikitaved/15/orig -> origin/gh/nikitaved/15/orig 2025-12-04T09:21:04.3178012Z * [new branch] gh/nikitaved/16/base -> origin/gh/nikitaved/16/base 2025-12-04T09:21:04.3179196Z * [new branch] gh/nikitaved/16/head -> origin/gh/nikitaved/16/head 2025-12-04T09:21:04.3180357Z * [new branch] gh/nikitaved/16/orig -> origin/gh/nikitaved/16/orig 2025-12-04T09:21:04.3182017Z * [new branch] gh/nikitaved/2/base -> origin/gh/nikitaved/2/base 2025-12-04T09:21:04.3183206Z * [new branch] gh/nikitaved/2/head -> origin/gh/nikitaved/2/head 2025-12-04T09:21:04.3184404Z * [new branch] gh/nikitaved/2/orig -> origin/gh/nikitaved/2/orig 2025-12-04T09:21:04.3186026Z * [new branch] gh/nikitaved/4/base -> origin/gh/nikitaved/4/base 2025-12-04T09:21:04.3187614Z * [new branch] gh/nikitaved/4/head -> origin/gh/nikitaved/4/head 2025-12-04T09:21:04.3188834Z * [new branch] gh/nikitaved/4/orig -> origin/gh/nikitaved/4/orig 2025-12-04T09:21:04.3190589Z * [new branch] gh/nikitaved/5/base -> origin/gh/nikitaved/5/base 2025-12-04T09:21:04.3191845Z * [new branch] gh/nikitaved/5/head -> origin/gh/nikitaved/5/head 2025-12-04T09:21:04.3193195Z * [new branch] gh/nikitaved/5/orig -> origin/gh/nikitaved/5/orig 2025-12-04T09:21:04.3194642Z * [new branch] gh/nikitaved/6/base -> origin/gh/nikitaved/6/base 2025-12-04T09:21:04.3195850Z * [new branch] gh/nikitaved/6/head -> origin/gh/nikitaved/6/head 2025-12-04T09:21:04.3197045Z * [new branch] gh/nikitaved/6/orig -> origin/gh/nikitaved/6/orig 2025-12-04T09:21:04.3198716Z * [new branch] gh/nikitaved/8/base -> origin/gh/nikitaved/8/base 2025-12-04T09:21:04.3200020Z * [new branch] gh/nikitaved/8/head -> origin/gh/nikitaved/8/head 2025-12-04T09:21:04.3202557Z * [new branch] gh/nikitaved/8/orig -> origin/gh/nikitaved/8/orig 2025-12-04T09:21:04.3204166Z * [new branch] gh/nikitaved/9/base -> origin/gh/nikitaved/9/base 2025-12-04T09:21:04.3205475Z * [new branch] gh/nikitaved/9/head -> origin/gh/nikitaved/9/head 2025-12-04T09:21:04.3206677Z * [new branch] gh/nikitaved/9/orig -> origin/gh/nikitaved/9/orig 2025-12-04T09:21:04.3209211Z * [new branch] gh/oulgen/10/base -> origin/gh/oulgen/10/base 2025-12-04T09:21:04.3210479Z * [new branch] gh/oulgen/10/head -> origin/gh/oulgen/10/head 2025-12-04T09:21:04.3217485Z * [new branch] gh/oulgen/10/orig -> origin/gh/oulgen/10/orig 2025-12-04T09:21:04.3217739Z * [new branch] gh/oulgen/11/base -> origin/gh/oulgen/11/base 2025-12-04T09:21:04.3217995Z * [new branch] gh/oulgen/11/head -> origin/gh/oulgen/11/head 2025-12-04T09:21:04.3218254Z * [new branch] gh/oulgen/11/orig -> origin/gh/oulgen/11/orig 2025-12-04T09:21:04.3218512Z * [new branch] gh/oulgen/12/base -> origin/gh/oulgen/12/base 2025-12-04T09:21:04.3219234Z * [new branch] gh/oulgen/12/head -> origin/gh/oulgen/12/head 2025-12-04T09:21:04.3220376Z * [new branch] gh/oulgen/12/orig -> origin/gh/oulgen/12/orig 2025-12-04T09:21:04.3221998Z * [new branch] gh/oulgen/13/base -> origin/gh/oulgen/13/base 2025-12-04T09:21:04.3223217Z * [new branch] gh/oulgen/13/head -> origin/gh/oulgen/13/head 2025-12-04T09:21:04.3224478Z * [new branch] gh/oulgen/13/orig -> origin/gh/oulgen/13/orig 2025-12-04T09:21:04.3226095Z * [new branch] gh/oulgen/14/base -> origin/gh/oulgen/14/base 2025-12-04T09:21:04.3227491Z * [new branch] gh/oulgen/14/head -> origin/gh/oulgen/14/head 2025-12-04T09:21:04.3228741Z * [new branch] gh/oulgen/14/orig -> origin/gh/oulgen/14/orig 2025-12-04T09:21:04.3230350Z * [new branch] gh/oulgen/15/base -> origin/gh/oulgen/15/base 2025-12-04T09:21:04.3231824Z * [new branch] gh/oulgen/15/head -> origin/gh/oulgen/15/head 2025-12-04T09:21:04.3233077Z * [new branch] gh/oulgen/15/orig -> origin/gh/oulgen/15/orig 2025-12-04T09:21:04.3234620Z * [new branch] gh/oulgen/16/base -> origin/gh/oulgen/16/base 2025-12-04T09:21:04.3235797Z * [new branch] gh/oulgen/16/head -> origin/gh/oulgen/16/head 2025-12-04T09:21:04.3237017Z * [new branch] gh/oulgen/16/orig -> origin/gh/oulgen/16/orig 2025-12-04T09:21:04.3238735Z * [new branch] gh/oulgen/17/base -> origin/gh/oulgen/17/base 2025-12-04T09:21:04.3239946Z * [new branch] gh/oulgen/17/head -> origin/gh/oulgen/17/head 2025-12-04T09:21:04.3241241Z * [new branch] gh/oulgen/17/orig -> origin/gh/oulgen/17/orig 2025-12-04T09:21:04.3242826Z * [new branch] gh/oulgen/18/base -> origin/gh/oulgen/18/base 2025-12-04T09:21:04.3243996Z * [new branch] gh/oulgen/18/head -> origin/gh/oulgen/18/head 2025-12-04T09:21:04.3245296Z * [new branch] gh/oulgen/18/orig -> origin/gh/oulgen/18/orig 2025-12-04T09:21:04.3246710Z * [new branch] gh/oulgen/19/base -> origin/gh/oulgen/19/base 2025-12-04T09:21:04.3247955Z * [new branch] gh/oulgen/19/head -> origin/gh/oulgen/19/head 2025-12-04T09:21:04.3249144Z * [new branch] gh/oulgen/19/orig -> origin/gh/oulgen/19/orig 2025-12-04T09:21:04.3250763Z * [new branch] gh/oulgen/20/base -> origin/gh/oulgen/20/base 2025-12-04T09:21:04.3251960Z * [new branch] gh/oulgen/20/head -> origin/gh/oulgen/20/head 2025-12-04T09:21:04.3253711Z * [new branch] gh/oulgen/20/orig -> origin/gh/oulgen/20/orig 2025-12-04T09:21:04.3255208Z * [new branch] gh/oulgen/21/base -> origin/gh/oulgen/21/base 2025-12-04T09:21:04.3256403Z * [new branch] gh/oulgen/21/head -> origin/gh/oulgen/21/head 2025-12-04T09:21:04.3257666Z * [new branch] gh/oulgen/21/orig -> origin/gh/oulgen/21/orig 2025-12-04T09:21:04.3259233Z * [new branch] gh/oulgen/22/base -> origin/gh/oulgen/22/base 2025-12-04T09:21:04.3260399Z * [new branch] gh/oulgen/22/head -> origin/gh/oulgen/22/head 2025-12-04T09:21:04.3262028Z * [new branch] gh/oulgen/22/orig -> origin/gh/oulgen/22/orig 2025-12-04T09:21:04.3263622Z * [new branch] gh/oulgen/23/base -> origin/gh/oulgen/23/base 2025-12-04T09:21:04.3264853Z * [new branch] gh/oulgen/23/head -> origin/gh/oulgen/23/head 2025-12-04T09:21:04.3266067Z * [new branch] gh/oulgen/23/orig -> origin/gh/oulgen/23/orig 2025-12-04T09:21:04.3268094Z * [new branch] gh/oulgen/24/base -> origin/gh/oulgen/24/base 2025-12-04T09:21:04.3269271Z * [new branch] gh/oulgen/24/head -> origin/gh/oulgen/24/head 2025-12-04T09:21:04.3270499Z * [new branch] gh/oulgen/24/orig -> origin/gh/oulgen/24/orig 2025-12-04T09:21:04.3272085Z * [new branch] gh/oulgen/25/base -> origin/gh/oulgen/25/base 2025-12-04T09:21:04.3273306Z * [new branch] gh/oulgen/25/head -> origin/gh/oulgen/25/head 2025-12-04T09:21:04.3279673Z * [new branch] gh/oulgen/25/orig -> origin/gh/oulgen/25/orig 2025-12-04T09:21:04.3280150Z * [new branch] gh/oulgen/26/base -> origin/gh/oulgen/26/base 2025-12-04T09:21:04.3280402Z * [new branch] gh/oulgen/26/head -> origin/gh/oulgen/26/head 2025-12-04T09:21:04.3280657Z * [new branch] gh/oulgen/26/orig -> origin/gh/oulgen/26/orig 2025-12-04T09:21:04.3281082Z * [new branch] gh/oulgen/4/base -> origin/gh/oulgen/4/base 2025-12-04T09:21:04.3282355Z * [new branch] gh/oulgen/4/head -> origin/gh/oulgen/4/head 2025-12-04T09:21:04.3283605Z * [new branch] gh/oulgen/4/orig -> origin/gh/oulgen/4/orig 2025-12-04T09:21:04.3285604Z * [new branch] gh/oulgen/7/base -> origin/gh/oulgen/7/base 2025-12-04T09:21:04.3286997Z * [new branch] gh/oulgen/7/head -> origin/gh/oulgen/7/head 2025-12-04T09:21:04.3288126Z * [new branch] gh/oulgen/7/orig -> origin/gh/oulgen/7/orig 2025-12-04T09:21:04.3289826Z * [new branch] gh/oulgen/8/base -> origin/gh/oulgen/8/base 2025-12-04T09:21:04.3291039Z * [new branch] gh/oulgen/8/head -> origin/gh/oulgen/8/head 2025-12-04T09:21:04.3292796Z * [new branch] gh/oulgen/8/orig -> origin/gh/oulgen/8/orig 2025-12-04T09:21:04.3294333Z * [new branch] gh/oulgen/9/base -> origin/gh/oulgen/9/base 2025-12-04T09:21:04.3295487Z * [new branch] gh/oulgen/9/head -> origin/gh/oulgen/9/head 2025-12-04T09:21:04.3296717Z * [new branch] gh/oulgen/9/orig -> origin/gh/oulgen/9/orig 2025-12-04T09:21:04.3298366Z * [new branch] gh/patvig/mtia-serialization -> origin/gh/patvig/mtia-serialization 2025-12-04T09:21:04.3300368Z * [new branch] gh/pearu/108/base -> origin/gh/pearu/108/base 2025-12-04T09:21:04.3302581Z * [new branch] gh/pearu/108/head -> origin/gh/pearu/108/head 2025-12-04T09:21:04.3304227Z * [new branch] gh/pearu/108/orig -> origin/gh/pearu/108/orig 2025-12-04T09:21:04.3306357Z * [new branch] gh/pearu/109/base -> origin/gh/pearu/109/base 2025-12-04T09:21:04.3307830Z * [new branch] gh/pearu/109/head -> origin/gh/pearu/109/head 2025-12-04T09:21:04.3309033Z * [new branch] gh/pearu/109/orig -> origin/gh/pearu/109/orig 2025-12-04T09:21:04.3310815Z * [new branch] gh/pearu/110/base -> origin/gh/pearu/110/base 2025-12-04T09:21:04.3312043Z * [new branch] gh/pearu/110/head -> origin/gh/pearu/110/head 2025-12-04T09:21:04.3313265Z * [new branch] gh/pearu/110/orig -> origin/gh/pearu/110/orig 2025-12-04T09:21:04.3314929Z * [new branch] gh/pearu/111/base -> origin/gh/pearu/111/base 2025-12-04T09:21:04.3316110Z * [new branch] gh/pearu/111/head -> origin/gh/pearu/111/head 2025-12-04T09:21:04.3317374Z * [new branch] gh/pearu/111/orig -> origin/gh/pearu/111/orig 2025-12-04T09:21:04.3319142Z * [new branch] gh/pearu/112/base -> origin/gh/pearu/112/base 2025-12-04T09:21:04.3320348Z * [new branch] gh/pearu/112/head -> origin/gh/pearu/112/head 2025-12-04T09:21:04.3321560Z * [new branch] gh/pearu/112/orig -> origin/gh/pearu/112/orig 2025-12-04T09:21:04.3323059Z * [new branch] gh/pearu/115/base -> origin/gh/pearu/115/base 2025-12-04T09:21:04.3324216Z * [new branch] gh/pearu/115/head -> origin/gh/pearu/115/head 2025-12-04T09:21:04.3325411Z * [new branch] gh/pearu/115/orig -> origin/gh/pearu/115/orig 2025-12-04T09:21:04.3326984Z * [new branch] gh/pearu/116/base -> origin/gh/pearu/116/base 2025-12-04T09:21:04.3328156Z * [new branch] gh/pearu/116/head -> origin/gh/pearu/116/head 2025-12-04T09:21:04.3329322Z * [new branch] gh/pearu/116/orig -> origin/gh/pearu/116/orig 2025-12-04T09:21:04.3330941Z * [new branch] gh/pearu/117/base -> origin/gh/pearu/117/base 2025-12-04T09:21:04.3332152Z * [new branch] gh/pearu/117/head -> origin/gh/pearu/117/head 2025-12-04T09:21:04.3333421Z * [new branch] gh/pearu/117/orig -> origin/gh/pearu/117/orig 2025-12-04T09:21:04.3334987Z * [new branch] gh/pearu/118/base -> origin/gh/pearu/118/base 2025-12-04T09:21:04.3336170Z * [new branch] gh/pearu/118/head -> origin/gh/pearu/118/head 2025-12-04T09:21:04.3342204Z * [new branch] gh/pearu/118/orig -> origin/gh/pearu/118/orig 2025-12-04T09:21:04.3344482Z * [new branch] gh/pearu/119/base -> origin/gh/pearu/119/base 2025-12-04T09:21:04.3345907Z * [new branch] gh/pearu/119/head -> origin/gh/pearu/119/head 2025-12-04T09:21:04.3347087Z * [new branch] gh/pearu/119/orig -> origin/gh/pearu/119/orig 2025-12-04T09:21:04.3348915Z * [new branch] gh/pearu/139/base -> origin/gh/pearu/139/base 2025-12-04T09:21:04.3350146Z * [new branch] gh/pearu/139/head -> origin/gh/pearu/139/head 2025-12-04T09:21:04.3351361Z * [new branch] gh/pearu/139/orig -> origin/gh/pearu/139/orig 2025-12-04T09:21:04.3353052Z * [new branch] gh/pearu/140/base -> origin/gh/pearu/140/base 2025-12-04T09:21:04.3354388Z * [new branch] gh/pearu/140/head -> origin/gh/pearu/140/head 2025-12-04T09:21:04.3355548Z * [new branch] gh/pearu/140/orig -> origin/gh/pearu/140/orig 2025-12-04T09:21:04.3357191Z * [new branch] gh/pearu/142/base -> origin/gh/pearu/142/base 2025-12-04T09:21:04.3358549Z * [new branch] gh/pearu/142/head -> origin/gh/pearu/142/head 2025-12-04T09:21:04.3359741Z * [new branch] gh/pearu/142/orig -> origin/gh/pearu/142/orig 2025-12-04T09:21:04.3361325Z * [new branch] gh/pearu/143/base -> origin/gh/pearu/143/base 2025-12-04T09:21:04.3362502Z * [new branch] gh/pearu/143/head -> origin/gh/pearu/143/head 2025-12-04T09:21:04.3363753Z * [new branch] gh/pearu/143/orig -> origin/gh/pearu/143/orig 2025-12-04T09:21:04.3365472Z * [new branch] gh/pearu/147/base -> origin/gh/pearu/147/base 2025-12-04T09:21:04.3366965Z * [new branch] gh/pearu/147/head -> origin/gh/pearu/147/head 2025-12-04T09:21:04.3367769Z * [new branch] gh/pearu/147/orig -> origin/gh/pearu/147/orig 2025-12-04T09:21:04.3369366Z * [new branch] gh/pearu/149/base -> origin/gh/pearu/149/base 2025-12-04T09:21:04.3370563Z * [new branch] gh/pearu/149/head -> origin/gh/pearu/149/head 2025-12-04T09:21:04.3371731Z * [new branch] gh/pearu/149/orig -> origin/gh/pearu/149/orig 2025-12-04T09:21:04.3373743Z * [new branch] gh/pearu/150/base -> origin/gh/pearu/150/base 2025-12-04T09:21:04.3374987Z * [new branch] gh/pearu/150/head -> origin/gh/pearu/150/head 2025-12-04T09:21:04.3376222Z * [new branch] gh/pearu/150/orig -> origin/gh/pearu/150/orig 2025-12-04T09:21:04.3377878Z * [new branch] gh/pearu/151/base -> origin/gh/pearu/151/base 2025-12-04T09:21:04.3379054Z * [new branch] gh/pearu/151/head -> origin/gh/pearu/151/head 2025-12-04T09:21:04.3380235Z * [new branch] gh/pearu/151/orig -> origin/gh/pearu/151/orig 2025-12-04T09:21:04.3381974Z * [new branch] gh/pearu/152/base -> origin/gh/pearu/152/base 2025-12-04T09:21:04.3383173Z * [new branch] gh/pearu/152/head -> origin/gh/pearu/152/head 2025-12-04T09:21:04.3384356Z * [new branch] gh/pearu/152/orig -> origin/gh/pearu/152/orig 2025-12-04T09:21:04.3385960Z * [new branch] gh/pearu/153/base -> origin/gh/pearu/153/base 2025-12-04T09:21:04.3387445Z * [new branch] gh/pearu/153/head -> origin/gh/pearu/153/head 2025-12-04T09:21:04.3388711Z * [new branch] gh/pearu/153/orig -> origin/gh/pearu/153/orig 2025-12-04T09:21:04.3390382Z * [new branch] gh/pearu/154/base -> origin/gh/pearu/154/base 2025-12-04T09:21:04.3391581Z * [new branch] gh/pearu/154/head -> origin/gh/pearu/154/head 2025-12-04T09:21:04.3392787Z * [new branch] gh/pearu/154/orig -> origin/gh/pearu/154/orig 2025-12-04T09:21:04.3394503Z * [new branch] gh/pearu/155/base -> origin/gh/pearu/155/base 2025-12-04T09:21:04.3395708Z * [new branch] gh/pearu/155/head -> origin/gh/pearu/155/head 2025-12-04T09:21:04.3396926Z * [new branch] gh/pearu/155/orig -> origin/gh/pearu/155/orig 2025-12-04T09:21:04.3398710Z * [new branch] gh/pearu/156/base -> origin/gh/pearu/156/base 2025-12-04T09:21:04.3405280Z * [new branch] gh/pearu/156/head -> origin/gh/pearu/156/head 2025-12-04T09:21:04.3406725Z * [new branch] gh/pearu/156/orig -> origin/gh/pearu/156/orig 2025-12-04T09:21:04.3408840Z * [new branch] gh/pearu/56/base -> origin/gh/pearu/56/base 2025-12-04T09:21:04.3410467Z * [new branch] gh/pearu/56/head -> origin/gh/pearu/56/head 2025-12-04T09:21:04.3411602Z * [new branch] gh/pearu/56/orig -> origin/gh/pearu/56/orig 2025-12-04T09:21:04.3413615Z * [new branch] gh/pearu/97/base -> origin/gh/pearu/97/base 2025-12-04T09:21:04.3414870Z * [new branch] gh/pearu/97/head -> origin/gh/pearu/97/head 2025-12-04T09:21:04.3416057Z * [new branch] gh/pearu/97/orig -> origin/gh/pearu/97/orig 2025-12-04T09:21:04.3417969Z * [new branch] gh/pianpwk/21/base -> origin/gh/pianpwk/21/base 2025-12-04T09:21:04.3419187Z * [new branch] gh/pianpwk/21/head -> origin/gh/pianpwk/21/head 2025-12-04T09:21:04.3420975Z * [new branch] gh/pianpwk/28/base -> origin/gh/pianpwk/28/base 2025-12-04T09:21:04.3422172Z * [new branch] gh/pianpwk/28/head -> origin/gh/pianpwk/28/head 2025-12-04T09:21:04.3423419Z * [new branch] gh/pianpwk/28/orig -> origin/gh/pianpwk/28/orig 2025-12-04T09:21:04.3425005Z * [new branch] gh/pianpwk/29/base -> origin/gh/pianpwk/29/base 2025-12-04T09:21:04.3426361Z * [new branch] gh/pianpwk/29/head -> origin/gh/pianpwk/29/head 2025-12-04T09:21:04.3428074Z * [new branch] gh/pianpwk/29/orig -> origin/gh/pianpwk/29/orig 2025-12-04T09:21:04.3429716Z * [new branch] gh/pianpwk/30/base -> origin/gh/pianpwk/30/base 2025-12-04T09:21:04.3430933Z * [new branch] gh/pianpwk/30/head -> origin/gh/pianpwk/30/head 2025-12-04T09:21:04.3432177Z * [new branch] gh/pianpwk/30/orig -> origin/gh/pianpwk/30/orig 2025-12-04T09:21:04.3433834Z * [new branch] gh/pianpwk/31/base -> origin/gh/pianpwk/31/base 2025-12-04T09:21:04.3435064Z * [new branch] gh/pianpwk/31/head -> origin/gh/pianpwk/31/head 2025-12-04T09:21:04.3436317Z * [new branch] gh/pianpwk/31/orig -> origin/gh/pianpwk/31/orig 2025-12-04T09:21:04.3437852Z * [new branch] gh/pianpwk/32/base -> origin/gh/pianpwk/32/base 2025-12-04T09:21:04.3439219Z * [new branch] gh/pianpwk/32/head -> origin/gh/pianpwk/32/head 2025-12-04T09:21:04.3440411Z * [new branch] gh/pianpwk/32/orig -> origin/gh/pianpwk/32/orig 2025-12-04T09:21:04.3441903Z * [new branch] gh/pianpwk/33/base -> origin/gh/pianpwk/33/base 2025-12-04T09:21:04.3443063Z * [new branch] gh/pianpwk/33/head -> origin/gh/pianpwk/33/head 2025-12-04T09:21:04.3444224Z * [new branch] gh/pianpwk/33/orig -> origin/gh/pianpwk/33/orig 2025-12-04T09:21:04.3446141Z * [new branch] gh/pianpwk/34/base -> origin/gh/pianpwk/34/base 2025-12-04T09:21:04.3447677Z * [new branch] gh/pianpwk/34/head -> origin/gh/pianpwk/34/head 2025-12-04T09:21:04.3449030Z * [new branch] gh/pianpwk/34/orig -> origin/gh/pianpwk/34/orig 2025-12-04T09:21:04.3450593Z * [new branch] gh/pianpwk/35/base -> origin/gh/pianpwk/35/base 2025-12-04T09:21:04.3452227Z * [new branch] gh/pianpwk/35/head -> origin/gh/pianpwk/35/head 2025-12-04T09:21:04.3453537Z * [new branch] gh/pianpwk/35/orig -> origin/gh/pianpwk/35/orig 2025-12-04T09:21:04.3455487Z * [new branch] gh/rec/141/base -> origin/gh/rec/141/base 2025-12-04T09:21:04.3456692Z * [new branch] gh/rec/141/head -> origin/gh/rec/141/head 2025-12-04T09:21:04.3458242Z * [new branch] gh/rec/153/base -> origin/gh/rec/153/base 2025-12-04T09:21:04.3459415Z * [new branch] gh/rec/153/head -> origin/gh/rec/153/head 2025-12-04T09:21:04.3460710Z * [new branch] gh/rec/153/orig -> origin/gh/rec/153/orig 2025-12-04T09:21:04.3467143Z * [new branch] gh/rec/154/base -> origin/gh/rec/154/base 2025-12-04T09:21:04.3468098Z * [new branch] gh/rec/154/head -> origin/gh/rec/154/head 2025-12-04T09:21:04.3468338Z * [new branch] gh/rec/154/orig -> origin/gh/rec/154/orig 2025-12-04T09:21:04.3468584Z * [new branch] gh/rec/164/base -> origin/gh/rec/164/base 2025-12-04T09:21:04.3468815Z * [new branch] gh/rec/164/head -> origin/gh/rec/164/head 2025-12-04T09:21:04.3469583Z * [new branch] gh/rec/164/orig -> origin/gh/rec/164/orig 2025-12-04T09:21:04.3471330Z * [new branch] gh/rec/166/base -> origin/gh/rec/166/base 2025-12-04T09:21:04.3472539Z * [new branch] gh/rec/166/head -> origin/gh/rec/166/head 2025-12-04T09:21:04.3473786Z * [new branch] gh/rec/166/orig -> origin/gh/rec/166/orig 2025-12-04T09:21:04.3475924Z * [new branch] gh/rec/167/base -> origin/gh/rec/167/base 2025-12-04T09:21:04.3477135Z * [new branch] gh/rec/167/head -> origin/gh/rec/167/head 2025-12-04T09:21:04.3478399Z * [new branch] gh/rec/167/orig -> origin/gh/rec/167/orig 2025-12-04T09:21:04.3480075Z * [new branch] gh/rec/168/base -> origin/gh/rec/168/base 2025-12-04T09:21:04.3481706Z * [new branch] gh/rec/168/head -> origin/gh/rec/168/head 2025-12-04T09:21:04.3482583Z * [new branch] gh/rec/168/orig -> origin/gh/rec/168/orig 2025-12-04T09:21:04.3484276Z * [new branch] gh/rec/169/base -> origin/gh/rec/169/base 2025-12-04T09:21:04.3485336Z * [new branch] gh/rec/169/head -> origin/gh/rec/169/head 2025-12-04T09:21:04.3486826Z * [new branch] gh/rec/169/orig -> origin/gh/rec/169/orig 2025-12-04T09:21:04.3489044Z * [new branch] gh/rec/170/base -> origin/gh/rec/170/base 2025-12-04T09:21:04.3490113Z * [new branch] gh/rec/170/head -> origin/gh/rec/170/head 2025-12-04T09:21:04.3491323Z * [new branch] gh/rec/170/orig -> origin/gh/rec/170/orig 2025-12-04T09:21:04.3493010Z * [new branch] gh/rec/171/base -> origin/gh/rec/171/base 2025-12-04T09:21:04.3494138Z * [new branch] gh/rec/171/head -> origin/gh/rec/171/head 2025-12-04T09:21:04.3495375Z * [new branch] gh/rec/171/orig -> origin/gh/rec/171/orig 2025-12-04T09:21:04.3497029Z * [new branch] gh/rec/172/base -> origin/gh/rec/172/base 2025-12-04T09:21:04.3498210Z * [new branch] gh/rec/172/head -> origin/gh/rec/172/head 2025-12-04T09:21:04.3499193Z * [new branch] gh/rec/172/orig -> origin/gh/rec/172/orig 2025-12-04T09:21:04.3501555Z * [new branch] gh/rec/173/base -> origin/gh/rec/173/base 2025-12-04T09:21:04.3502890Z * [new branch] gh/rec/173/head -> origin/gh/rec/173/head 2025-12-04T09:21:04.3504134Z * [new branch] gh/rec/173/orig -> origin/gh/rec/173/orig 2025-12-04T09:21:04.3505975Z * [new branch] gh/rec/174/base -> origin/gh/rec/174/base 2025-12-04T09:21:04.3507214Z * [new branch] gh/rec/174/head -> origin/gh/rec/174/head 2025-12-04T09:21:04.3508453Z * [new branch] gh/rec/174/orig -> origin/gh/rec/174/orig 2025-12-04T09:21:04.3510708Z * [new branch] gh/rec/175/base -> origin/gh/rec/175/base 2025-12-04T09:21:04.3511777Z * [new branch] gh/rec/175/head -> origin/gh/rec/175/head 2025-12-04T09:21:04.3512978Z * [new branch] gh/rec/175/orig -> origin/gh/rec/175/orig 2025-12-04T09:21:04.3514911Z * [new branch] gh/rec/176/base -> origin/gh/rec/176/base 2025-12-04T09:21:04.3515819Z * [new branch] gh/rec/176/head -> origin/gh/rec/176/head 2025-12-04T09:21:04.3517020Z * [new branch] gh/rec/176/orig -> origin/gh/rec/176/orig 2025-12-04T09:21:04.3518805Z * [new branch] gh/rec/177/base -> origin/gh/rec/177/base 2025-12-04T09:21:04.3519920Z * [new branch] gh/rec/177/head -> origin/gh/rec/177/head 2025-12-04T09:21:04.3521161Z * [new branch] gh/rec/177/orig -> origin/gh/rec/177/orig 2025-12-04T09:21:04.3523832Z * [new branch] gh/robert-hardwick/3/base -> origin/gh/robert-hardwick/3/base 2025-12-04T09:21:04.3530786Z * [new branch] gh/robert-hardwick/3/head -> origin/gh/robert-hardwick/3/head 2025-12-04T09:21:04.3531344Z * [new branch] gh/robert-hardwick/3/orig -> origin/gh/robert-hardwick/3/orig 2025-12-04T09:21:04.3531888Z * [new branch] gh/robert-hardwick/4/base -> origin/gh/robert-hardwick/4/base 2025-12-04T09:21:04.3532420Z * [new branch] gh/robert-hardwick/4/head -> origin/gh/robert-hardwick/4/head 2025-12-04T09:21:04.3532889Z * [new branch] gh/robert-hardwick/4/orig -> origin/gh/robert-hardwick/4/orig 2025-12-04T09:21:04.3533409Z * [new branch] gh/robert-hardwick/5/base -> origin/gh/robert-hardwick/5/base 2025-12-04T09:21:04.3533946Z * [new branch] gh/robert-hardwick/5/head -> origin/gh/robert-hardwick/5/head 2025-12-04T09:21:04.3535138Z * [new branch] gh/robert-hardwick/5/orig -> origin/gh/robert-hardwick/5/orig 2025-12-04T09:21:04.3536806Z * [new branch] gh/robert-hardwick/6/base -> origin/gh/robert-hardwick/6/base 2025-12-04T09:21:04.3537961Z * [new branch] gh/robert-hardwick/6/head -> origin/gh/robert-hardwick/6/head 2025-12-04T09:21:04.3539268Z * [new branch] gh/robert-hardwick/6/orig -> origin/gh/robert-hardwick/6/orig 2025-12-04T09:21:04.3541255Z * [new branch] gh/robert-hardwick/7/base -> origin/gh/robert-hardwick/7/base 2025-12-04T09:21:04.3542313Z * [new branch] gh/robert-hardwick/7/head -> origin/gh/robert-hardwick/7/head 2025-12-04T09:21:04.3543548Z * [new branch] gh/robert-hardwick/7/orig -> origin/gh/robert-hardwick/7/orig 2025-12-04T09:21:04.3545140Z * [new branch] gh/robert-hardwick/8/base -> origin/gh/robert-hardwick/8/base 2025-12-04T09:21:04.3546331Z * [new branch] gh/robert-hardwick/8/head -> origin/gh/robert-hardwick/8/head 2025-12-04T09:21:04.3547894Z * [new branch] gh/robert-hardwick/8/orig -> origin/gh/robert-hardwick/8/orig 2025-12-04T09:21:04.3549518Z * [new branch] gh/robert-hardwick/9/base -> origin/gh/robert-hardwick/9/base 2025-12-04T09:21:04.3550735Z * [new branch] gh/robert-hardwick/9/head -> origin/gh/robert-hardwick/9/head 2025-12-04T09:21:04.3552027Z * [new branch] gh/robert-hardwick/9/orig -> origin/gh/robert-hardwick/9/orig 2025-12-04T09:21:04.3554391Z * [new branch] gh/rtimpe/1/base -> origin/gh/rtimpe/1/base 2025-12-04T09:21:04.3555772Z * [new branch] gh/rtimpe/1/head -> origin/gh/rtimpe/1/head 2025-12-04T09:21:04.3557413Z * [new branch] gh/rtimpe/2/base -> origin/gh/rtimpe/2/base 2025-12-04T09:21:04.3558591Z * [new branch] gh/rtimpe/2/head -> origin/gh/rtimpe/2/head 2025-12-04T09:21:04.3560687Z * [new branch] gh/rtimpe/22/base -> origin/gh/rtimpe/22/base 2025-12-04T09:21:04.3561907Z * [new branch] gh/rtimpe/22/head -> origin/gh/rtimpe/22/head 2025-12-04T09:21:04.3563087Z * [new branch] gh/rtimpe/22/orig -> origin/gh/rtimpe/22/orig 2025-12-04T09:21:04.3564625Z * [new branch] gh/rtimpe/23/base -> origin/gh/rtimpe/23/base 2025-12-04T09:21:04.3565919Z * [new branch] gh/rtimpe/23/head -> origin/gh/rtimpe/23/head 2025-12-04T09:21:04.3567090Z * [new branch] gh/rtimpe/23/orig -> origin/gh/rtimpe/23/orig 2025-12-04T09:21:04.3568646Z * [new branch] gh/rtimpe/24/base -> origin/gh/rtimpe/24/base 2025-12-04T09:21:04.3569860Z * [new branch] gh/rtimpe/24/head -> origin/gh/rtimpe/24/head 2025-12-04T09:21:04.3571016Z * [new branch] gh/rtimpe/24/orig -> origin/gh/rtimpe/24/orig 2025-12-04T09:21:04.3572703Z * [new branch] gh/rtimpe/25/base -> origin/gh/rtimpe/25/base 2025-12-04T09:21:04.3573881Z * [new branch] gh/rtimpe/25/head -> origin/gh/rtimpe/25/head 2025-12-04T09:21:04.3575098Z * [new branch] gh/rtimpe/25/orig -> origin/gh/rtimpe/25/orig 2025-12-04T09:21:04.3576641Z * [new branch] gh/rtimpe/26/base -> origin/gh/rtimpe/26/base 2025-12-04T09:21:04.3577845Z * [new branch] gh/rtimpe/26/head -> origin/gh/rtimpe/26/head 2025-12-04T09:21:04.3579045Z * [new branch] gh/rtimpe/26/orig -> origin/gh/rtimpe/26/orig 2025-12-04T09:21:04.3580529Z * [new branch] gh/rtimpe/27/base -> origin/gh/rtimpe/27/base 2025-12-04T09:21:04.3582161Z * [new branch] gh/rtimpe/27/head -> origin/gh/rtimpe/27/head 2025-12-04T09:21:04.3583384Z * [new branch] gh/rtimpe/27/orig -> origin/gh/rtimpe/27/orig 2025-12-04T09:21:04.3584972Z * [new branch] gh/rtimpe/28/base -> origin/gh/rtimpe/28/base 2025-12-04T09:21:04.3586213Z * [new branch] gh/rtimpe/28/head -> origin/gh/rtimpe/28/head 2025-12-04T09:21:04.3593494Z * [new branch] gh/rtimpe/28/orig -> origin/gh/rtimpe/28/orig 2025-12-04T09:21:04.3595209Z * [new branch] gh/rtimpe/29/base -> origin/gh/rtimpe/29/base 2025-12-04T09:21:04.3596405Z * [new branch] gh/rtimpe/29/head -> origin/gh/rtimpe/29/head 2025-12-04T09:21:04.3597699Z * [new branch] gh/rtimpe/29/orig -> origin/gh/rtimpe/29/orig 2025-12-04T09:21:04.3599254Z * [new branch] gh/rtimpe/3/base -> origin/gh/rtimpe/3/base 2025-12-04T09:21:04.3600405Z * [new branch] gh/rtimpe/3/head -> origin/gh/rtimpe/3/head 2025-12-04T09:21:04.3602672Z * [new branch] gh/rtimpe/30/base -> origin/gh/rtimpe/30/base 2025-12-04T09:21:04.3603897Z * [new branch] gh/rtimpe/30/head -> origin/gh/rtimpe/30/head 2025-12-04T09:21:04.3605119Z * [new branch] gh/rtimpe/30/orig -> origin/gh/rtimpe/30/orig 2025-12-04T09:21:04.3607218Z * [new branch] gh/rtimpe/31/base -> origin/gh/rtimpe/31/base 2025-12-04T09:21:04.3608453Z * [new branch] gh/rtimpe/31/head -> origin/gh/rtimpe/31/head 2025-12-04T09:21:04.3609769Z * [new branch] gh/rtimpe/31/orig -> origin/gh/rtimpe/31/orig 2025-12-04T09:21:04.3611496Z * [new branch] gh/rtimpe/32/base -> origin/gh/rtimpe/32/base 2025-12-04T09:21:04.3612696Z * [new branch] gh/rtimpe/32/head -> origin/gh/rtimpe/32/head 2025-12-04T09:21:04.3613989Z * [new branch] gh/rtimpe/32/orig -> origin/gh/rtimpe/32/orig 2025-12-04T09:21:04.3615583Z * [new branch] gh/rtimpe/33/base -> origin/gh/rtimpe/33/base 2025-12-04T09:21:04.3616799Z * [new branch] gh/rtimpe/33/head -> origin/gh/rtimpe/33/head 2025-12-04T09:21:04.3618057Z * [new branch] gh/rtimpe/33/orig -> origin/gh/rtimpe/33/orig 2025-12-04T09:21:04.3619498Z * [new branch] gh/rtimpe/34/base -> origin/gh/rtimpe/34/base 2025-12-04T09:21:04.3620665Z * [new branch] gh/rtimpe/34/head -> origin/gh/rtimpe/34/head 2025-12-04T09:21:04.3621960Z * [new branch] gh/rtimpe/34/orig -> origin/gh/rtimpe/34/orig 2025-12-04T09:21:04.3623506Z * [new branch] gh/rtimpe/35/base -> origin/gh/rtimpe/35/base 2025-12-04T09:21:04.3624754Z * [new branch] gh/rtimpe/35/head -> origin/gh/rtimpe/35/head 2025-12-04T09:21:04.3625961Z * [new branch] gh/rtimpe/35/orig -> origin/gh/rtimpe/35/orig 2025-12-04T09:21:04.3628007Z * [new branch] gh/rtimpe/4/base -> origin/gh/rtimpe/4/base 2025-12-04T09:21:04.3629218Z * [new branch] gh/rtimpe/4/head -> origin/gh/rtimpe/4/head 2025-12-04T09:21:04.3631287Z * [new branch] gh/ruisizhang123/1/base -> origin/gh/ruisizhang123/1/base 2025-12-04T09:21:04.3632555Z * [new branch] gh/ruisizhang123/1/head -> origin/gh/ruisizhang123/1/head 2025-12-04T09:21:04.3633753Z * [new branch] gh/ruisizhang123/1/orig -> origin/gh/ruisizhang123/1/orig 2025-12-04T09:21:04.3635549Z * [new branch] gh/ruisizhang123/4/base -> origin/gh/ruisizhang123/4/base 2025-12-04T09:21:04.3636799Z * [new branch] gh/ruisizhang123/4/head -> origin/gh/ruisizhang123/4/head 2025-12-04T09:21:04.3638001Z * [new branch] gh/ruisizhang123/4/orig -> origin/gh/ruisizhang123/4/orig 2025-12-04T09:21:04.3639709Z * [new branch] gh/ruisizhang123/5/base -> origin/gh/ruisizhang123/5/base 2025-12-04T09:21:04.3640912Z * [new branch] gh/ruisizhang123/5/head -> origin/gh/ruisizhang123/5/head 2025-12-04T09:21:04.3642145Z * [new branch] gh/ruisizhang123/5/orig -> origin/gh/ruisizhang123/5/orig 2025-12-04T09:21:04.3643780Z * [new branch] gh/ruisizhang123/6/base -> origin/gh/ruisizhang123/6/base 2025-12-04T09:21:04.3644970Z * [new branch] gh/ruisizhang123/6/head -> origin/gh/ruisizhang123/6/head 2025-12-04T09:21:04.3646138Z * [new branch] gh/ruisizhang123/6/orig -> origin/gh/ruisizhang123/6/orig 2025-12-04T09:21:04.3647773Z * [new branch] gh/ruisizhang123/7/base -> origin/gh/ruisizhang123/7/base 2025-12-04T09:21:04.3648972Z * [new branch] gh/ruisizhang123/7/head -> origin/gh/ruisizhang123/7/head 2025-12-04T09:21:04.3655660Z * [new branch] gh/ruisizhang123/7/orig -> origin/gh/ruisizhang123/7/orig 2025-12-04T09:21:04.3657266Z * [new branch] gh/ruisizhang123/8/base -> origin/gh/ruisizhang123/8/base 2025-12-04T09:21:04.3658461Z * [new branch] gh/ruisizhang123/8/head -> origin/gh/ruisizhang123/8/head 2025-12-04T09:21:04.3659709Z * [new branch] gh/ruisizhang123/8/orig -> origin/gh/ruisizhang123/8/orig 2025-12-04T09:21:04.3661339Z * [new branch] gh/ruisizhang123/9/base -> origin/gh/ruisizhang123/9/base 2025-12-04T09:21:04.3662692Z * [new branch] gh/ruisizhang123/9/head -> origin/gh/ruisizhang123/9/head 2025-12-04T09:21:04.3663888Z * [new branch] gh/ruisizhang123/9/orig -> origin/gh/ruisizhang123/9/orig 2025-12-04T09:21:04.3666010Z * [new branch] gh/seemethere/52/base -> origin/gh/seemethere/52/base 2025-12-04T09:21:04.3667588Z * [new branch] gh/seemethere/52/head -> origin/gh/seemethere/52/head 2025-12-04T09:21:04.3669321Z * [new branch] gh/seemethere/52/orig -> origin/gh/seemethere/52/orig 2025-12-04T09:21:04.3670993Z * [new branch] gh/seemethere/53/base -> origin/gh/seemethere/53/base 2025-12-04T09:21:04.3672246Z * [new branch] gh/seemethere/53/head -> origin/gh/seemethere/53/head 2025-12-04T09:21:04.3673484Z * [new branch] gh/seemethere/53/orig -> origin/gh/seemethere/53/orig 2025-12-04T09:21:04.3675399Z * [new branch] gh/seemethere/54/base -> origin/gh/seemethere/54/base 2025-12-04T09:21:04.3676648Z * [new branch] gh/seemethere/54/head -> origin/gh/seemethere/54/head 2025-12-04T09:21:04.3677982Z * [new branch] gh/seemethere/54/orig -> origin/gh/seemethere/54/orig 2025-12-04T09:21:04.3679533Z * [new branch] gh/seemethere/55/base -> origin/gh/seemethere/55/base 2025-12-04T09:21:04.3680716Z * [new branch] gh/seemethere/55/head -> origin/gh/seemethere/55/head 2025-12-04T09:21:04.3681937Z * [new branch] gh/seemethere/55/orig -> origin/gh/seemethere/55/orig 2025-12-04T09:21:04.3683545Z * [new branch] gh/seemethere/59/base -> origin/gh/seemethere/59/base 2025-12-04T09:21:04.3684770Z * [new branch] gh/seemethere/59/head -> origin/gh/seemethere/59/head 2025-12-04T09:21:04.3685948Z * [new branch] gh/seemethere/59/orig -> origin/gh/seemethere/59/orig 2025-12-04T09:21:04.3687533Z * [new branch] gh/seemethere/62/base -> origin/gh/seemethere/62/base 2025-12-04T09:21:04.3688720Z * [new branch] gh/seemethere/62/head -> origin/gh/seemethere/62/head 2025-12-04T09:21:04.3689931Z * [new branch] gh/seemethere/62/orig -> origin/gh/seemethere/62/orig 2025-12-04T09:21:04.3691466Z * [new branch] gh/seemethere/63/base -> origin/gh/seemethere/63/base 2025-12-04T09:21:04.3692628Z * [new branch] gh/seemethere/63/head -> origin/gh/seemethere/63/head 2025-12-04T09:21:04.3693844Z * [new branch] gh/seemethere/63/orig -> origin/gh/seemethere/63/orig 2025-12-04T09:21:04.3695387Z * [new branch] gh/seemethere/71/base -> origin/gh/seemethere/71/base 2025-12-04T09:21:04.3696585Z * [new branch] gh/seemethere/71/head -> origin/gh/seemethere/71/head 2025-12-04T09:21:04.3697734Z * [new branch] gh/seemethere/71/orig -> origin/gh/seemethere/71/orig 2025-12-04T09:21:04.3699487Z * [new branch] gh/seemethere/72/base -> origin/gh/seemethere/72/base 2025-12-04T09:21:04.3700644Z * [new branch] gh/seemethere/72/head -> origin/gh/seemethere/72/head 2025-12-04T09:21:04.3702720Z * [new branch] gh/seemethere/72/orig -> origin/gh/seemethere/72/orig 2025-12-04T09:21:04.3704339Z * [new branch] gh/seemethere/73/base -> origin/gh/seemethere/73/base 2025-12-04T09:21:04.3705564Z * [new branch] gh/seemethere/73/head -> origin/gh/seemethere/73/head 2025-12-04T09:21:04.3706993Z * [new branch] gh/seemethere/73/orig -> origin/gh/seemethere/73/orig 2025-12-04T09:21:04.3708663Z * [new branch] gh/seemethere/74/base -> origin/gh/seemethere/74/base 2025-12-04T09:21:04.3709859Z * [new branch] gh/seemethere/74/head -> origin/gh/seemethere/74/head 2025-12-04T09:21:04.3711097Z * [new branch] gh/seemethere/74/orig -> origin/gh/seemethere/74/orig 2025-12-04T09:21:04.3717783Z * [new branch] gh/seemethere/75/base -> origin/gh/seemethere/75/base 2025-12-04T09:21:04.3718240Z * [new branch] gh/seemethere/75/head -> origin/gh/seemethere/75/head 2025-12-04T09:21:04.3718662Z * [new branch] gh/seemethere/75/orig -> origin/gh/seemethere/75/orig 2025-12-04T09:21:04.3719034Z * [new branch] gh/seemethere/76/base -> origin/gh/seemethere/76/base 2025-12-04T09:21:04.3719314Z * [new branch] gh/seemethere/76/head -> origin/gh/seemethere/76/head 2025-12-04T09:21:04.3719699Z * [new branch] gh/seemethere/76/orig -> origin/gh/seemethere/76/orig 2025-12-04T09:21:04.3722041Z * [new branch] gh/shunting314/145/base -> origin/gh/shunting314/145/base 2025-12-04T09:21:04.3723343Z * [new branch] gh/shunting314/145/head -> origin/gh/shunting314/145/head 2025-12-04T09:21:04.3724688Z * [new branch] gh/shunting314/145/orig -> origin/gh/shunting314/145/orig 2025-12-04T09:21:04.3726643Z * [new branch] gh/shunting314/176/base -> origin/gh/shunting314/176/base 2025-12-04T09:21:04.3728203Z * [new branch] gh/shunting314/176/head -> origin/gh/shunting314/176/head 2025-12-04T09:21:04.3729397Z * [new branch] gh/shunting314/176/orig -> origin/gh/shunting314/176/orig 2025-12-04T09:21:04.3731078Z * [new branch] gh/shunting314/249/base -> origin/gh/shunting314/249/base 2025-12-04T09:21:04.3732380Z * [new branch] gh/shunting314/249/head -> origin/gh/shunting314/249/head 2025-12-04T09:21:04.3733810Z * [new branch] gh/shunting314/249/orig -> origin/gh/shunting314/249/orig 2025-12-04T09:21:04.3735522Z * [new branch] gh/shunting314/253/base -> origin/gh/shunting314/253/base 2025-12-04T09:21:04.3736661Z * [new branch] gh/shunting314/253/head -> origin/gh/shunting314/253/head 2025-12-04T09:21:04.3737826Z * [new branch] gh/shunting314/253/orig -> origin/gh/shunting314/253/orig 2025-12-04T09:21:04.3739419Z * [new branch] gh/shunting314/256/base -> origin/gh/shunting314/256/base 2025-12-04T09:21:04.3740626Z * [new branch] gh/shunting314/256/head -> origin/gh/shunting314/256/head 2025-12-04T09:21:04.3741857Z * [new branch] gh/shunting314/256/orig -> origin/gh/shunting314/256/orig 2025-12-04T09:21:04.3743819Z * [new branch] gh/shunting314/257/base -> origin/gh/shunting314/257/base 2025-12-04T09:21:04.3745066Z * [new branch] gh/shunting314/257/head -> origin/gh/shunting314/257/head 2025-12-04T09:21:04.3746235Z * [new branch] gh/shunting314/257/orig -> origin/gh/shunting314/257/orig 2025-12-04T09:21:04.3748521Z * [new branch] gh/shunting314/258/base -> origin/gh/shunting314/258/base 2025-12-04T09:21:04.3749580Z * [new branch] gh/shunting314/258/head -> origin/gh/shunting314/258/head 2025-12-04T09:21:04.3750967Z * [new branch] gh/shunting314/258/orig -> origin/gh/shunting314/258/orig 2025-12-04T09:21:04.3752493Z * [new branch] gh/shunting314/259/base -> origin/gh/shunting314/259/base 2025-12-04T09:21:04.3753738Z * [new branch] gh/shunting314/259/head -> origin/gh/shunting314/259/head 2025-12-04T09:21:04.3754938Z * [new branch] gh/shunting314/259/orig -> origin/gh/shunting314/259/orig 2025-12-04T09:21:04.3757144Z * [new branch] gh/shunting314/260/base -> origin/gh/shunting314/260/base 2025-12-04T09:21:04.3758609Z * [new branch] gh/shunting314/260/head -> origin/gh/shunting314/260/head 2025-12-04T09:21:04.3759963Z * [new branch] gh/shunting314/260/orig -> origin/gh/shunting314/260/orig 2025-12-04T09:21:04.3761665Z * [new branch] gh/shunting314/261/base -> origin/gh/shunting314/261/base 2025-12-04T09:21:04.3762963Z * [new branch] gh/shunting314/261/head -> origin/gh/shunting314/261/head 2025-12-04T09:21:04.3764151Z * [new branch] gh/shunting314/261/orig -> origin/gh/shunting314/261/orig 2025-12-04T09:21:04.3765876Z * [new branch] gh/shunting314/262/base -> origin/gh/shunting314/262/base 2025-12-04T09:21:04.3767123Z * [new branch] gh/shunting314/262/head -> origin/gh/shunting314/262/head 2025-12-04T09:21:04.3768494Z * [new branch] gh/shunting314/262/orig -> origin/gh/shunting314/262/orig 2025-12-04T09:21:04.3770181Z * [new branch] gh/shunting314/263/base -> origin/gh/shunting314/263/base 2025-12-04T09:21:04.3771560Z * [new branch] gh/shunting314/263/head -> origin/gh/shunting314/263/head 2025-12-04T09:21:04.3772763Z * [new branch] gh/shunting314/263/orig -> origin/gh/shunting314/263/orig 2025-12-04T09:21:04.3774495Z * [new branch] gh/shunting314/264/base -> origin/gh/shunting314/264/base 2025-12-04T09:21:04.3780677Z * [new branch] gh/shunting314/264/head -> origin/gh/shunting314/264/head 2025-12-04T09:21:04.3780961Z * [new branch] gh/shunting314/264/orig -> origin/gh/shunting314/264/orig 2025-12-04T09:21:04.3781253Z * [new branch] gh/shunting314/265/base -> origin/gh/shunting314/265/base 2025-12-04T09:21:04.3781527Z * [new branch] gh/shunting314/265/head -> origin/gh/shunting314/265/head 2025-12-04T09:21:04.3781822Z * [new branch] gh/shunting314/265/orig -> origin/gh/shunting314/265/orig 2025-12-04T09:21:04.3783492Z * [new branch] gh/shunting314/266/base -> origin/gh/shunting314/266/base 2025-12-04T09:21:04.3784893Z * [new branch] gh/shunting314/266/head -> origin/gh/shunting314/266/head 2025-12-04T09:21:04.3786239Z * [new branch] gh/shunting314/266/orig -> origin/gh/shunting314/266/orig 2025-12-04T09:21:04.3788394Z * [new branch] gh/shunting314/267/base -> origin/gh/shunting314/267/base 2025-12-04T09:21:04.3789853Z * [new branch] gh/shunting314/267/head -> origin/gh/shunting314/267/head 2025-12-04T09:21:04.3791128Z * [new branch] gh/shunting314/267/orig -> origin/gh/shunting314/267/orig 2025-12-04T09:21:04.3793225Z * [new branch] gh/shunting314/268/base -> origin/gh/shunting314/268/base 2025-12-04T09:21:04.3794565Z * [new branch] gh/shunting314/268/head -> origin/gh/shunting314/268/head 2025-12-04T09:21:04.3795786Z * [new branch] gh/shunting314/268/orig -> origin/gh/shunting314/268/orig 2025-12-04T09:21:04.3797530Z * [new branch] gh/shunting314/269/base -> origin/gh/shunting314/269/base 2025-12-04T09:21:04.3798850Z * [new branch] gh/shunting314/269/head -> origin/gh/shunting314/269/head 2025-12-04T09:21:04.3800153Z * [new branch] gh/shunting314/269/orig -> origin/gh/shunting314/269/orig 2025-12-04T09:21:04.3803341Z * [new branch] gh/silverguo/1/base -> origin/gh/silverguo/1/base 2025-12-04T09:21:04.3804756Z * [new branch] gh/silverguo/1/head -> origin/gh/silverguo/1/head 2025-12-04T09:21:04.3806225Z * [new branch] gh/silverguo/2/base -> origin/gh/silverguo/2/base 2025-12-04T09:21:04.3807344Z * [new branch] gh/silverguo/2/head -> origin/gh/silverguo/2/head 2025-12-04T09:21:04.3809230Z * [new branch] gh/silverguo/3/base -> origin/gh/silverguo/3/base 2025-12-04T09:21:04.3810494Z * [new branch] gh/silverguo/3/head -> origin/gh/silverguo/3/head 2025-12-04T09:21:04.3811943Z * [new branch] gh/silverguo/4/base -> origin/gh/silverguo/4/base 2025-12-04T09:21:04.3813221Z * [new branch] gh/silverguo/4/head -> origin/gh/silverguo/4/head 2025-12-04T09:21:04.3815220Z * [new branch] gh/slayton58/39/base -> origin/gh/slayton58/39/base 2025-12-04T09:21:04.3816437Z * [new branch] gh/slayton58/39/head -> origin/gh/slayton58/39/head 2025-12-04T09:21:04.3817672Z * [new branch] gh/slayton58/39/orig -> origin/gh/slayton58/39/orig 2025-12-04T09:21:04.3819439Z * [new branch] gh/slayton58/42/base -> origin/gh/slayton58/42/base 2025-12-04T09:21:04.3820621Z * [new branch] gh/slayton58/42/head -> origin/gh/slayton58/42/head 2025-12-04T09:21:04.3821963Z * [new branch] gh/slayton58/42/orig -> origin/gh/slayton58/42/orig 2025-12-04T09:21:04.3823510Z * [new branch] gh/slayton58/43/base -> origin/gh/slayton58/43/base 2025-12-04T09:21:04.3824674Z * [new branch] gh/slayton58/43/head -> origin/gh/slayton58/43/head 2025-12-04T09:21:04.3826391Z * [new branch] gh/slayton58/43/orig -> origin/gh/slayton58/43/orig 2025-12-04T09:21:04.3828534Z * [new branch] gh/slayton58/44/base -> origin/gh/slayton58/44/base 2025-12-04T09:21:04.3829827Z * [new branch] gh/slayton58/44/head -> origin/gh/slayton58/44/head 2025-12-04T09:21:04.3831028Z * [new branch] gh/slayton58/44/orig -> origin/gh/slayton58/44/orig 2025-12-04T09:21:04.3832694Z * [new branch] gh/slayton58/45/base -> origin/gh/slayton58/45/base 2025-12-04T09:21:04.3833939Z * [new branch] gh/slayton58/45/head -> origin/gh/slayton58/45/head 2025-12-04T09:21:04.3835174Z * [new branch] gh/slayton58/45/orig -> origin/gh/slayton58/45/orig 2025-12-04T09:21:04.3837050Z * [new branch] gh/slayton58/46/base -> origin/gh/slayton58/46/base 2025-12-04T09:21:04.3843317Z * [new branch] gh/slayton58/46/head -> origin/gh/slayton58/46/head 2025-12-04T09:21:04.3844622Z * [new branch] gh/slayton58/46/orig -> origin/gh/slayton58/46/orig 2025-12-04T09:21:04.3846276Z * [new branch] gh/slayton58/6/base -> origin/gh/slayton58/6/base 2025-12-04T09:21:04.3847710Z * [new branch] gh/slayton58/6/head -> origin/gh/slayton58/6/head 2025-12-04T09:21:04.3849288Z * [new branch] gh/slayton58/7/base -> origin/gh/slayton58/7/base 2025-12-04T09:21:04.3850431Z * [new branch] gh/slayton58/7/head -> origin/gh/slayton58/7/head 2025-12-04T09:21:04.3852588Z * [new branch] gh/soulitzer/269/base -> origin/gh/soulitzer/269/base 2025-12-04T09:21:04.3853705Z * [new branch] gh/soulitzer/269/head -> origin/gh/soulitzer/269/head 2025-12-04T09:21:04.3854981Z * [new branch] gh/soulitzer/269/orig -> origin/gh/soulitzer/269/orig 2025-12-04T09:21:04.3856732Z * [new branch] gh/soulitzer/276/base -> origin/gh/soulitzer/276/base 2025-12-04T09:21:04.3858090Z * [new branch] gh/soulitzer/276/head -> origin/gh/soulitzer/276/head 2025-12-04T09:21:04.3859302Z * [new branch] gh/soulitzer/276/orig -> origin/gh/soulitzer/276/orig 2025-12-04T09:21:04.3861212Z * [new branch] gh/soulitzer/287/base -> origin/gh/soulitzer/287/base 2025-12-04T09:21:04.3862375Z * [new branch] gh/soulitzer/287/head -> origin/gh/soulitzer/287/head 2025-12-04T09:21:04.3863646Z * [new branch] gh/soulitzer/287/orig -> origin/gh/soulitzer/287/orig 2025-12-04T09:21:04.3865372Z * [new branch] gh/soulitzer/296/base -> origin/gh/soulitzer/296/base 2025-12-04T09:21:04.3866768Z * [new branch] gh/soulitzer/296/head -> origin/gh/soulitzer/296/head 2025-12-04T09:21:04.3868283Z * [new branch] gh/soulitzer/296/orig -> origin/gh/soulitzer/296/orig 2025-12-04T09:21:04.3869979Z * [new branch] gh/soulitzer/299/base -> origin/gh/soulitzer/299/base 2025-12-04T09:21:04.3871259Z * [new branch] gh/soulitzer/299/head -> origin/gh/soulitzer/299/head 2025-12-04T09:21:04.3872503Z * [new branch] gh/soulitzer/299/orig -> origin/gh/soulitzer/299/orig 2025-12-04T09:21:04.3874227Z * [new branch] gh/soulitzer/300/base -> origin/gh/soulitzer/300/base 2025-12-04T09:21:04.3875637Z * [new branch] gh/soulitzer/300/head -> origin/gh/soulitzer/300/head 2025-12-04T09:21:04.3876863Z * [new branch] gh/soulitzer/300/orig -> origin/gh/soulitzer/300/orig 2025-12-04T09:21:04.3878672Z * [new branch] gh/soulitzer/301/base -> origin/gh/soulitzer/301/base 2025-12-04T09:21:04.3880063Z * [new branch] gh/soulitzer/301/head -> origin/gh/soulitzer/301/head 2025-12-04T09:21:04.3881293Z * [new branch] gh/soulitzer/301/orig -> origin/gh/soulitzer/301/orig 2025-12-04T09:21:04.3882864Z * [new branch] gh/soulitzer/313/base -> origin/gh/soulitzer/313/base 2025-12-04T09:21:04.3884039Z * [new branch] gh/soulitzer/313/head -> origin/gh/soulitzer/313/head 2025-12-04T09:21:04.3885347Z * [new branch] gh/soulitzer/313/orig -> origin/gh/soulitzer/313/orig 2025-12-04T09:21:04.3886919Z * [new branch] gh/soulitzer/319/base -> origin/gh/soulitzer/319/base 2025-12-04T09:21:04.3888117Z * [new branch] gh/soulitzer/319/head -> origin/gh/soulitzer/319/head 2025-12-04T09:21:04.3889321Z * [new branch] gh/soulitzer/319/orig -> origin/gh/soulitzer/319/orig 2025-12-04T09:21:04.3891027Z * [new branch] gh/soulitzer/320/base -> origin/gh/soulitzer/320/base 2025-12-04T09:21:04.3892340Z * [new branch] gh/soulitzer/320/head -> origin/gh/soulitzer/320/head 2025-12-04T09:21:04.3893546Z * [new branch] gh/soulitzer/320/orig -> origin/gh/soulitzer/320/orig 2025-12-04T09:21:04.3895413Z * [new branch] gh/soulitzer/336/base -> origin/gh/soulitzer/336/base 2025-12-04T09:21:04.3896530Z * [new branch] gh/soulitzer/336/head -> origin/gh/soulitzer/336/head 2025-12-04T09:21:04.3897666Z * [new branch] gh/soulitzer/336/orig -> origin/gh/soulitzer/336/orig 2025-12-04T09:21:04.3899429Z * [new branch] gh/soulitzer/347/base -> origin/gh/soulitzer/347/base 2025-12-04T09:21:04.3910771Z * [new branch] gh/soulitzer/347/head -> origin/gh/soulitzer/347/head 2025-12-04T09:21:04.3911072Z * [new branch] gh/soulitzer/347/orig -> origin/gh/soulitzer/347/orig 2025-12-04T09:21:04.3911359Z * [new branch] gh/soulitzer/349/base -> origin/gh/soulitzer/349/base 2025-12-04T09:21:04.3911713Z * [new branch] gh/soulitzer/349/head -> origin/gh/soulitzer/349/head 2025-12-04T09:21:04.3912127Z * [new branch] gh/soulitzer/349/orig -> origin/gh/soulitzer/349/orig 2025-12-04T09:21:04.3913692Z * [new branch] gh/soulitzer/350/base -> origin/gh/soulitzer/350/base 2025-12-04T09:21:04.3914984Z * [new branch] gh/soulitzer/350/head -> origin/gh/soulitzer/350/head 2025-12-04T09:21:04.3916240Z * [new branch] gh/soulitzer/350/orig -> origin/gh/soulitzer/350/orig 2025-12-04T09:21:04.3917930Z * [new branch] gh/soulitzer/351/base -> origin/gh/soulitzer/351/base 2025-12-04T09:21:04.3919248Z * [new branch] gh/soulitzer/351/head -> origin/gh/soulitzer/351/head 2025-12-04T09:21:04.3920426Z * [new branch] gh/soulitzer/351/orig -> origin/gh/soulitzer/351/orig 2025-12-04T09:21:04.3922042Z * [new branch] gh/soulitzer/353/base -> origin/gh/soulitzer/353/base 2025-12-04T09:21:04.3923353Z * [new branch] gh/soulitzer/353/head -> origin/gh/soulitzer/353/head 2025-12-04T09:21:04.3925041Z * [new branch] gh/soulitzer/353/orig -> origin/gh/soulitzer/353/orig 2025-12-04T09:21:04.3927386Z * [new branch] gh/soulitzer/358/base -> origin/gh/soulitzer/358/base 2025-12-04T09:21:04.3928692Z * [new branch] gh/soulitzer/358/head -> origin/gh/soulitzer/358/head 2025-12-04T09:21:04.3929910Z * [new branch] gh/soulitzer/358/orig -> origin/gh/soulitzer/358/orig 2025-12-04T09:21:04.3932127Z * [new branch] gh/soulitzer/359/base -> origin/gh/soulitzer/359/base 2025-12-04T09:21:04.3933464Z * [new branch] gh/soulitzer/359/head -> origin/gh/soulitzer/359/head 2025-12-04T09:21:04.3934741Z * [new branch] gh/soulitzer/359/orig -> origin/gh/soulitzer/359/orig 2025-12-04T09:21:04.3936800Z * [new branch] gh/soulitzer/374/base -> origin/gh/soulitzer/374/base 2025-12-04T09:21:04.3938107Z * [new branch] gh/soulitzer/374/head -> origin/gh/soulitzer/374/head 2025-12-04T09:21:04.3939252Z * [new branch] gh/soulitzer/374/orig -> origin/gh/soulitzer/374/orig 2025-12-04T09:21:04.3940834Z * [new branch] gh/soulitzer/375/base -> origin/gh/soulitzer/375/base 2025-12-04T09:21:04.3942037Z * [new branch] gh/soulitzer/375/head -> origin/gh/soulitzer/375/head 2025-12-04T09:21:04.3943168Z * [new branch] gh/soulitzer/375/orig -> origin/gh/soulitzer/375/orig 2025-12-04T09:21:04.3944707Z * [new branch] gh/soulitzer/380/base -> origin/gh/soulitzer/380/base 2025-12-04T09:21:04.3945935Z * [new branch] gh/soulitzer/380/head -> origin/gh/soulitzer/380/head 2025-12-04T09:21:04.3947530Z * [new branch] gh/soulitzer/380/orig -> origin/gh/soulitzer/380/orig 2025-12-04T09:21:04.3949128Z * [new branch] gh/soulitzer/385/base -> origin/gh/soulitzer/385/base 2025-12-04T09:21:04.3950494Z * [new branch] gh/soulitzer/385/head -> origin/gh/soulitzer/385/head 2025-12-04T09:21:04.3951770Z * [new branch] gh/soulitzer/385/orig -> origin/gh/soulitzer/385/orig 2025-12-04T09:21:04.3953466Z * [new branch] gh/soulitzer/386/base -> origin/gh/soulitzer/386/base 2025-12-04T09:21:04.3954710Z * [new branch] gh/soulitzer/386/head -> origin/gh/soulitzer/386/head 2025-12-04T09:21:04.3955916Z * [new branch] gh/soulitzer/386/orig -> origin/gh/soulitzer/386/orig 2025-12-04T09:21:04.3957554Z * [new branch] gh/soulitzer/387/base -> origin/gh/soulitzer/387/base 2025-12-04T09:21:04.3958797Z * [new branch] gh/soulitzer/387/head -> origin/gh/soulitzer/387/head 2025-12-04T09:21:04.3960102Z * [new branch] gh/soulitzer/387/orig -> origin/gh/soulitzer/387/orig 2025-12-04T09:21:04.3961775Z * [new branch] gh/soulitzer/388/base -> origin/gh/soulitzer/388/base 2025-12-04T09:21:04.3963455Z * [new branch] gh/soulitzer/388/head -> origin/gh/soulitzer/388/head 2025-12-04T09:21:04.3967729Z * [new branch] gh/soulitzer/388/orig -> origin/gh/soulitzer/388/orig 2025-12-04T09:21:04.3968206Z * [new branch] gh/soulitzer/389/base -> origin/gh/soulitzer/389/base 2025-12-04T09:21:04.3968655Z * [new branch] gh/soulitzer/389/head -> origin/gh/soulitzer/389/head 2025-12-04T09:21:04.3969545Z * [new branch] gh/soulitzer/389/orig -> origin/gh/soulitzer/389/orig 2025-12-04T09:21:04.3971312Z * [new branch] gh/soulitzer/390/base -> origin/gh/soulitzer/390/base 2025-12-04T09:21:04.3972489Z * [new branch] gh/soulitzer/390/head -> origin/gh/soulitzer/390/head 2025-12-04T09:21:04.3973587Z * [new branch] gh/soulitzer/390/orig -> origin/gh/soulitzer/390/orig 2025-12-04T09:21:04.3975207Z * [new branch] gh/soulitzer/391/base -> origin/gh/soulitzer/391/base 2025-12-04T09:21:04.3976378Z * [new branch] gh/soulitzer/391/head -> origin/gh/soulitzer/391/head 2025-12-04T09:21:04.3977569Z * [new branch] gh/soulitzer/391/orig -> origin/gh/soulitzer/391/orig 2025-12-04T09:21:04.3979262Z * [new branch] gh/soulitzer/392/base -> origin/gh/soulitzer/392/base 2025-12-04T09:21:04.3980574Z * [new branch] gh/soulitzer/392/head -> origin/gh/soulitzer/392/head 2025-12-04T09:21:04.3981741Z * [new branch] gh/soulitzer/392/orig -> origin/gh/soulitzer/392/orig 2025-12-04T09:21:04.3983794Z * [new branch] gh/swolchok/728/next -> origin/gh/swolchok/728/next 2025-12-04T09:21:04.3985722Z * [new branch] gh/swolchok/819/base -> origin/gh/swolchok/819/base 2025-12-04T09:21:04.3987147Z * [new branch] gh/swolchok/819/head -> origin/gh/swolchok/819/head 2025-12-04T09:21:04.3988435Z * [new branch] gh/swolchok/819/orig -> origin/gh/swolchok/819/orig 2025-12-04T09:21:04.3990048Z * [new branch] gh/swolchok/824/base -> origin/gh/swolchok/824/base 2025-12-04T09:21:04.3991357Z * [new branch] gh/swolchok/824/head -> origin/gh/swolchok/824/head 2025-12-04T09:21:04.3992551Z * [new branch] gh/swolchok/824/orig -> origin/gh/swolchok/824/orig 2025-12-04T09:21:04.3994804Z * [new branch] gh/swolchok/829/base -> origin/gh/swolchok/829/base 2025-12-04T09:21:04.3995952Z * [new branch] gh/swolchok/829/head -> origin/gh/swolchok/829/head 2025-12-04T09:21:04.3997186Z * [new branch] gh/swolchok/829/orig -> origin/gh/swolchok/829/orig 2025-12-04T09:21:04.3999100Z * [new branch] gh/swolchok/839/base -> origin/gh/swolchok/839/base 2025-12-04T09:21:04.4000247Z * [new branch] gh/swolchok/839/head -> origin/gh/swolchok/839/head 2025-12-04T09:21:04.4002288Z * [new branch] gh/swolchok/839/orig -> origin/gh/swolchok/839/orig 2025-12-04T09:21:04.4003981Z * [new branch] gh/swolchok/841/base -> origin/gh/swolchok/841/base 2025-12-04T09:21:04.4005224Z * [new branch] gh/swolchok/841/head -> origin/gh/swolchok/841/head 2025-12-04T09:21:04.4006484Z * [new branch] gh/swolchok/841/orig -> origin/gh/swolchok/841/orig 2025-12-04T09:21:04.4008131Z * [new branch] gh/swolchok/842/base -> origin/gh/swolchok/842/base 2025-12-04T09:21:04.4009656Z * [new branch] gh/swolchok/842/head -> origin/gh/swolchok/842/head 2025-12-04T09:21:04.4010830Z * [new branch] gh/swolchok/842/orig -> origin/gh/swolchok/842/orig 2025-12-04T09:21:04.4012517Z * [new branch] gh/swolchok/845/base -> origin/gh/swolchok/845/base 2025-12-04T09:21:04.4013791Z * [new branch] gh/swolchok/845/head -> origin/gh/swolchok/845/head 2025-12-04T09:21:04.4015228Z * [new branch] gh/swolchok/845/orig -> origin/gh/swolchok/845/orig 2025-12-04T09:21:04.4017287Z * [new branch] gh/swolchok/848/base -> origin/gh/swolchok/848/base 2025-12-04T09:21:04.4018580Z * [new branch] gh/swolchok/848/head -> origin/gh/swolchok/848/head 2025-12-04T09:21:04.4019882Z * [new branch] gh/swolchok/848/orig -> origin/gh/swolchok/848/orig 2025-12-04T09:21:04.4021448Z * [new branch] gh/swolchok/856/base -> origin/gh/swolchok/856/base 2025-12-04T09:21:04.4022622Z * [new branch] gh/swolchok/856/head -> origin/gh/swolchok/856/head 2025-12-04T09:21:04.4023807Z * [new branch] gh/swolchok/856/orig -> origin/gh/swolchok/856/orig 2025-12-04T09:21:04.4030467Z * [new branch] gh/swolchok/860/base -> origin/gh/swolchok/860/base 2025-12-04T09:21:04.4030746Z * [new branch] gh/swolchok/860/head -> origin/gh/swolchok/860/head 2025-12-04T09:21:04.4031009Z * [new branch] gh/swolchok/860/orig -> origin/gh/swolchok/860/orig 2025-12-04T09:21:04.4031281Z * [new branch] gh/swolchok/861/base -> origin/gh/swolchok/861/base 2025-12-04T09:21:04.4031738Z * [new branch] gh/swolchok/861/head -> origin/gh/swolchok/861/head 2025-12-04T09:21:04.4033094Z * [new branch] gh/swolchok/861/orig -> origin/gh/swolchok/861/orig 2025-12-04T09:21:04.4034795Z * [new branch] gh/swolchok/862/base -> origin/gh/swolchok/862/base 2025-12-04T09:21:04.4035958Z * [new branch] gh/swolchok/862/head -> origin/gh/swolchok/862/head 2025-12-04T09:21:04.4037262Z * [new branch] gh/swolchok/862/orig -> origin/gh/swolchok/862/orig 2025-12-04T09:21:04.4039265Z * [new branch] gh/swolchok/863/base -> origin/gh/swolchok/863/base 2025-12-04T09:21:04.4040424Z * [new branch] gh/swolchok/863/head -> origin/gh/swolchok/863/head 2025-12-04T09:21:04.4041775Z * [new branch] gh/swolchok/863/orig -> origin/gh/swolchok/863/orig 2025-12-04T09:21:04.4043470Z * [new branch] gh/swolchok/864/base -> origin/gh/swolchok/864/base 2025-12-04T09:21:04.4044582Z * [new branch] gh/swolchok/864/head -> origin/gh/swolchok/864/head 2025-12-04T09:21:04.4045813Z * [new branch] gh/swolchok/864/orig -> origin/gh/swolchok/864/orig 2025-12-04T09:21:04.4047436Z * [new branch] gh/swolchok/865/base -> origin/gh/swolchok/865/base 2025-12-04T09:21:04.4048784Z * [new branch] gh/swolchok/865/head -> origin/gh/swolchok/865/head 2025-12-04T09:21:04.4050021Z * [new branch] gh/swolchok/865/orig -> origin/gh/swolchok/865/orig 2025-12-04T09:21:04.4052147Z * [new branch] gh/swolchok/866/base -> origin/gh/swolchok/866/base 2025-12-04T09:21:04.4053495Z * [new branch] gh/swolchok/866/head -> origin/gh/swolchok/866/head 2025-12-04T09:21:04.4054790Z * [new branch] gh/swolchok/866/orig -> origin/gh/swolchok/866/orig 2025-12-04T09:21:04.4056431Z * [new branch] gh/swolchok/867/base -> origin/gh/swolchok/867/base 2025-12-04T09:21:04.4057634Z * [new branch] gh/swolchok/867/head -> origin/gh/swolchok/867/head 2025-12-04T09:21:04.4058869Z * [new branch] gh/swolchok/867/orig -> origin/gh/swolchok/867/orig 2025-12-04T09:21:04.4060485Z * [new branch] gh/swolchok/868/base -> origin/gh/swolchok/868/base 2025-12-04T09:21:04.4061734Z * [new branch] gh/swolchok/868/head -> origin/gh/swolchok/868/head 2025-12-04T09:21:04.4062942Z * [new branch] gh/swolchok/868/orig -> origin/gh/swolchok/868/orig 2025-12-04T09:21:04.4064574Z * [new branch] gh/swolchok/869/base -> origin/gh/swolchok/869/base 2025-12-04T09:21:04.4065816Z * [new branch] gh/swolchok/869/head -> origin/gh/swolchok/869/head 2025-12-04T09:21:04.4067170Z * [new branch] gh/swolchok/869/orig -> origin/gh/swolchok/869/orig 2025-12-04T09:21:04.4069184Z * [new branch] gh/swolchok/870/base -> origin/gh/swolchok/870/base 2025-12-04T09:21:04.4070338Z * [new branch] gh/swolchok/870/head -> origin/gh/swolchok/870/head 2025-12-04T09:21:04.4071630Z * [new branch] gh/swolchok/870/orig -> origin/gh/swolchok/870/orig 2025-12-04T09:21:04.4073350Z * [new branch] gh/swolchok/871/base -> origin/gh/swolchok/871/base 2025-12-04T09:21:04.4074661Z * [new branch] gh/swolchok/871/head -> origin/gh/swolchok/871/head 2025-12-04T09:21:04.4075962Z * [new branch] gh/swolchok/871/orig -> origin/gh/swolchok/871/orig 2025-12-04T09:21:04.4078500Z * [new branch] gh/teja-rao/4/base -> origin/gh/teja-rao/4/base 2025-12-04T09:21:04.4079929Z * [new branch] gh/teja-rao/4/head -> origin/gh/teja-rao/4/head 2025-12-04T09:21:04.4081193Z * [new branch] gh/teja-rao/4/orig -> origin/gh/teja-rao/4/orig 2025-12-04T09:21:04.4083653Z * [new branch] gh/tianyu-l/2/base -> origin/gh/tianyu-l/2/base 2025-12-04T09:21:04.4084907Z * [new branch] gh/tianyu-l/2/head -> origin/gh/tianyu-l/2/head 2025-12-04T09:21:04.4086127Z * [new branch] gh/tianyu-l/2/orig -> origin/gh/tianyu-l/2/orig 2025-12-04T09:21:04.4093454Z * [new branch] gh/tianyu-l/3/base -> origin/gh/tianyu-l/3/base 2025-12-04T09:21:04.4094717Z * [new branch] gh/tianyu-l/3/orig -> origin/gh/tianyu-l/3/orig 2025-12-04T09:21:04.4096323Z * [new branch] gh/tianyu-l/4/base -> origin/gh/tianyu-l/4/base 2025-12-04T09:21:04.4097563Z * [new branch] gh/tianyu-l/4/head -> origin/gh/tianyu-l/4/head 2025-12-04T09:21:04.4098812Z * [new branch] gh/tianyu-l/4/orig -> origin/gh/tianyu-l/4/orig 2025-12-04T09:21:04.4101994Z * [new branch] gh/tugsbayasgalan/10/base -> origin/gh/tugsbayasgalan/10/base 2025-12-04T09:21:04.4103309Z * [new branch] gh/tugsbayasgalan/10/head -> origin/gh/tugsbayasgalan/10/head 2025-12-04T09:21:04.4104597Z * [new branch] gh/tugsbayasgalan/10/orig -> origin/gh/tugsbayasgalan/10/orig 2025-12-04T09:21:04.4106326Z * [new branch] gh/tugsbayasgalan/13/base -> origin/gh/tugsbayasgalan/13/base 2025-12-04T09:21:04.4107676Z * [new branch] gh/tugsbayasgalan/13/head -> origin/gh/tugsbayasgalan/13/head 2025-12-04T09:21:04.4109334Z * [new branch] gh/tugsbayasgalan/13/orig -> origin/gh/tugsbayasgalan/13/orig 2025-12-04T09:21:04.4111243Z * [new branch] gh/tugsbayasgalan/17/base -> origin/gh/tugsbayasgalan/17/base 2025-12-04T09:21:04.4112388Z * [new branch] gh/tugsbayasgalan/17/head -> origin/gh/tugsbayasgalan/17/head 2025-12-04T09:21:04.4113641Z * [new branch] gh/tugsbayasgalan/17/orig -> origin/gh/tugsbayasgalan/17/orig 2025-12-04T09:21:04.4115361Z * [new branch] gh/tugsbayasgalan/2/base -> origin/gh/tugsbayasgalan/2/base 2025-12-04T09:21:04.4116626Z * [new branch] gh/tugsbayasgalan/2/head -> origin/gh/tugsbayasgalan/2/head 2025-12-04T09:21:04.4117875Z * [new branch] gh/tugsbayasgalan/2/orig -> origin/gh/tugsbayasgalan/2/orig 2025-12-04T09:21:04.4120372Z * [new branch] gh/tugsbayasgalan/28/base -> origin/gh/tugsbayasgalan/28/base 2025-12-04T09:21:04.4121564Z * [new branch] gh/tugsbayasgalan/28/head -> origin/gh/tugsbayasgalan/28/head 2025-12-04T09:21:04.4122763Z * [new branch] gh/tugsbayasgalan/28/orig -> origin/gh/tugsbayasgalan/28/orig 2025-12-04T09:21:04.4124659Z * [new branch] gh/tugsbayasgalan/32/base -> origin/gh/tugsbayasgalan/32/base 2025-12-04T09:21:04.4125872Z * [new branch] gh/tugsbayasgalan/32/head -> origin/gh/tugsbayasgalan/32/head 2025-12-04T09:21:04.4127088Z * [new branch] gh/tugsbayasgalan/32/orig -> origin/gh/tugsbayasgalan/32/orig 2025-12-04T09:21:04.4128882Z * [new branch] gh/tugsbayasgalan/35/base -> origin/gh/tugsbayasgalan/35/base 2025-12-04T09:21:04.4130193Z * [new branch] gh/tugsbayasgalan/35/head -> origin/gh/tugsbayasgalan/35/head 2025-12-04T09:21:04.4131369Z * [new branch] gh/tugsbayasgalan/35/orig -> origin/gh/tugsbayasgalan/35/orig 2025-12-04T09:21:04.4132965Z * [new branch] gh/tugsbayasgalan/36/base -> origin/gh/tugsbayasgalan/36/base 2025-12-04T09:21:04.4134193Z * [new branch] gh/tugsbayasgalan/36/head -> origin/gh/tugsbayasgalan/36/head 2025-12-04T09:21:04.4135400Z * [new branch] gh/tugsbayasgalan/36/orig -> origin/gh/tugsbayasgalan/36/orig 2025-12-04T09:21:04.4137034Z * [new branch] gh/tugsbayasgalan/37/base -> origin/gh/tugsbayasgalan/37/base 2025-12-04T09:21:04.4138194Z * [new branch] gh/tugsbayasgalan/37/head -> origin/gh/tugsbayasgalan/37/head 2025-12-04T09:21:04.4139369Z * [new branch] gh/tugsbayasgalan/37/orig -> origin/gh/tugsbayasgalan/37/orig 2025-12-04T09:21:04.4140905Z * [new branch] gh/tugsbayasgalan/43/base -> origin/gh/tugsbayasgalan/43/base 2025-12-04T09:21:04.4142159Z * [new branch] gh/tugsbayasgalan/43/head -> origin/gh/tugsbayasgalan/43/head 2025-12-04T09:21:04.4143324Z * [new branch] gh/tugsbayasgalan/43/orig -> origin/gh/tugsbayasgalan/43/orig 2025-12-04T09:21:04.4144880Z * [new branch] gh/tugsbayasgalan/48/base -> origin/gh/tugsbayasgalan/48/base 2025-12-04T09:21:04.4146075Z * [new branch] gh/tugsbayasgalan/48/head -> origin/gh/tugsbayasgalan/48/head 2025-12-04T09:21:04.4147628Z * [new branch] gh/tugsbayasgalan/48/orig -> origin/gh/tugsbayasgalan/48/orig 2025-12-04T09:21:04.4149305Z * [new branch] gh/tugsbayasgalan/51/base -> origin/gh/tugsbayasgalan/51/base 2025-12-04T09:21:04.4157256Z * [new branch] gh/tugsbayasgalan/51/head -> origin/gh/tugsbayasgalan/51/head 2025-12-04T09:21:04.4157594Z * [new branch] gh/tugsbayasgalan/51/orig -> origin/gh/tugsbayasgalan/51/orig 2025-12-04T09:21:04.4158200Z * [new branch] gh/tugsbayasgalan/52/base -> origin/gh/tugsbayasgalan/52/base 2025-12-04T09:21:04.4159997Z * [new branch] gh/tugsbayasgalan/52/head -> origin/gh/tugsbayasgalan/52/head 2025-12-04T09:21:04.4160918Z * [new branch] gh/tugsbayasgalan/52/orig -> origin/gh/tugsbayasgalan/52/orig 2025-12-04T09:21:04.4162509Z * [new branch] gh/tugsbayasgalan/53/base -> origin/gh/tugsbayasgalan/53/base 2025-12-04T09:21:04.4163741Z * [new branch] gh/tugsbayasgalan/53/head -> origin/gh/tugsbayasgalan/53/head 2025-12-04T09:21:04.4165070Z * [new branch] gh/tugsbayasgalan/53/orig -> origin/gh/tugsbayasgalan/53/orig 2025-12-04T09:21:04.4166932Z * [new branch] gh/tugsbayasgalan/55/base -> origin/gh/tugsbayasgalan/55/base 2025-12-04T09:21:04.4168285Z * [new branch] gh/tugsbayasgalan/55/head -> origin/gh/tugsbayasgalan/55/head 2025-12-04T09:21:04.4169522Z * [new branch] gh/tugsbayasgalan/55/orig -> origin/gh/tugsbayasgalan/55/orig 2025-12-04T09:21:04.4171292Z * [new branch] gh/tugsbayasgalan/59/base -> origin/gh/tugsbayasgalan/59/base 2025-12-04T09:21:04.4173467Z * [new branch] gh/tugsbayasgalan/59/head -> origin/gh/tugsbayasgalan/59/head 2025-12-04T09:21:04.4174663Z * [new branch] gh/tugsbayasgalan/59/orig -> origin/gh/tugsbayasgalan/59/orig 2025-12-04T09:21:04.4176228Z * [new branch] gh/tugsbayasgalan/6/base -> origin/gh/tugsbayasgalan/6/base 2025-12-04T09:21:04.4177445Z * [new branch] gh/tugsbayasgalan/6/head -> origin/gh/tugsbayasgalan/6/head 2025-12-04T09:21:04.4178622Z * [new branch] gh/tugsbayasgalan/6/orig -> origin/gh/tugsbayasgalan/6/orig 2025-12-04T09:21:04.4180088Z * [new branch] gh/tugsbayasgalan/60/base -> origin/gh/tugsbayasgalan/60/base 2025-12-04T09:21:04.4181317Z * [new branch] gh/tugsbayasgalan/60/head -> origin/gh/tugsbayasgalan/60/head 2025-12-04T09:21:04.4182550Z * [new branch] gh/tugsbayasgalan/60/orig -> origin/gh/tugsbayasgalan/60/orig 2025-12-04T09:21:04.4184620Z * [new branch] gh/tugsbayasgalan/61/base -> origin/gh/tugsbayasgalan/61/base 2025-12-04T09:21:04.4185792Z * [new branch] gh/tugsbayasgalan/61/head -> origin/gh/tugsbayasgalan/61/head 2025-12-04T09:21:04.4187341Z * [new branch] gh/tugsbayasgalan/61/orig -> origin/gh/tugsbayasgalan/61/orig 2025-12-04T09:21:04.4189254Z * [new branch] gh/tugsbayasgalan/63/base -> origin/gh/tugsbayasgalan/63/base 2025-12-04T09:21:04.4190492Z * [new branch] gh/tugsbayasgalan/63/head -> origin/gh/tugsbayasgalan/63/head 2025-12-04T09:21:04.4191706Z * [new branch] gh/tugsbayasgalan/63/orig -> origin/gh/tugsbayasgalan/63/orig 2025-12-04T09:21:04.4193335Z * [new branch] gh/tugsbayasgalan/67/base -> origin/gh/tugsbayasgalan/67/base 2025-12-04T09:21:04.4194597Z * [new branch] gh/tugsbayasgalan/67/head -> origin/gh/tugsbayasgalan/67/head 2025-12-04T09:21:04.4195810Z * [new branch] gh/tugsbayasgalan/67/orig -> origin/gh/tugsbayasgalan/67/orig 2025-12-04T09:21:04.4197730Z * [new branch] gh/tugsbayasgalan/68/base -> origin/gh/tugsbayasgalan/68/base 2025-12-04T09:21:04.4199085Z * [new branch] gh/tugsbayasgalan/68/head -> origin/gh/tugsbayasgalan/68/head 2025-12-04T09:21:04.4200529Z * [new branch] gh/tugsbayasgalan/68/orig -> origin/gh/tugsbayasgalan/68/orig 2025-12-04T09:21:04.4202969Z * [new branch] gh/tugsbayasgalan/7/base -> origin/gh/tugsbayasgalan/7/base 2025-12-04T09:21:04.4204209Z * [new branch] gh/tugsbayasgalan/7/head -> origin/gh/tugsbayasgalan/7/head 2025-12-04T09:21:04.4205608Z * [new branch] gh/tugsbayasgalan/7/orig -> origin/gh/tugsbayasgalan/7/orig 2025-12-04T09:21:04.4207576Z * [new branch] gh/tugsbayasgalan/70/base -> origin/gh/tugsbayasgalan/70/base 2025-12-04T09:21:04.4209046Z * [new branch] gh/tugsbayasgalan/70/head -> origin/gh/tugsbayasgalan/70/head 2025-12-04T09:21:04.4210312Z * [new branch] gh/tugsbayasgalan/70/orig -> origin/gh/tugsbayasgalan/70/orig 2025-12-04T09:21:04.4212574Z * [new branch] gh/tugsbayasgalan/71/base -> origin/gh/tugsbayasgalan/71/base 2025-12-04T09:21:04.4215000Z * [new branch] gh/tugsbayasgalan/71/head -> origin/gh/tugsbayasgalan/71/head 2025-12-04T09:21:04.4215310Z * [new branch] gh/tugsbayasgalan/71/orig -> origin/gh/tugsbayasgalan/71/orig 2025-12-04T09:21:04.4217579Z * [new branch] gh/tugsbayasgalan/72/base -> origin/gh/tugsbayasgalan/72/base 2025-12-04T09:21:04.4218255Z * [new branch] gh/tugsbayasgalan/72/head -> origin/gh/tugsbayasgalan/72/head 2025-12-04T09:21:04.4219611Z * [new branch] gh/tugsbayasgalan/72/orig -> origin/gh/tugsbayasgalan/72/orig 2025-12-04T09:21:04.4221470Z * [new branch] gh/tugsbayasgalan/73/base -> origin/gh/tugsbayasgalan/73/base 2025-12-04T09:21:04.4222861Z * [new branch] gh/tugsbayasgalan/73/head -> origin/gh/tugsbayasgalan/73/head 2025-12-04T09:21:04.4224081Z * [new branch] gh/tugsbayasgalan/73/orig -> origin/gh/tugsbayasgalan/73/orig 2025-12-04T09:21:04.4225999Z * [new branch] gh/tugsbayasgalan/74/base -> origin/gh/tugsbayasgalan/74/base 2025-12-04T09:21:04.4227354Z * [new branch] gh/tugsbayasgalan/74/head -> origin/gh/tugsbayasgalan/74/head 2025-12-04T09:21:04.4228715Z * [new branch] gh/tugsbayasgalan/74/orig -> origin/gh/tugsbayasgalan/74/orig 2025-12-04T09:21:04.4230417Z * [new branch] gh/tugsbayasgalan/75/base -> origin/gh/tugsbayasgalan/75/base 2025-12-04T09:21:04.4231651Z * [new branch] gh/tugsbayasgalan/75/head -> origin/gh/tugsbayasgalan/75/head 2025-12-04T09:21:04.4232866Z * [new branch] gh/tugsbayasgalan/75/orig -> origin/gh/tugsbayasgalan/75/orig 2025-12-04T09:21:04.4234388Z * [new branch] gh/tugsbayasgalan/76/base -> origin/gh/tugsbayasgalan/76/base 2025-12-04T09:21:04.4235707Z * [new branch] gh/tugsbayasgalan/76/head -> origin/gh/tugsbayasgalan/76/head 2025-12-04T09:21:04.4237142Z * [new branch] gh/tugsbayasgalan/76/orig -> origin/gh/tugsbayasgalan/76/orig 2025-12-04T09:21:04.4239167Z * [new branch] gh/tugsbayasgalan/77/base -> origin/gh/tugsbayasgalan/77/base 2025-12-04T09:21:04.4240361Z * [new branch] gh/tugsbayasgalan/77/head -> origin/gh/tugsbayasgalan/77/head 2025-12-04T09:21:04.4241490Z * [new branch] gh/tugsbayasgalan/77/orig -> origin/gh/tugsbayasgalan/77/orig 2025-12-04T09:21:04.4243238Z * [new branch] gh/tugsbayasgalan/78/base -> origin/gh/tugsbayasgalan/78/base 2025-12-04T09:21:04.4244549Z * [new branch] gh/tugsbayasgalan/78/head -> origin/gh/tugsbayasgalan/78/head 2025-12-04T09:21:04.4245765Z * [new branch] gh/tugsbayasgalan/78/orig -> origin/gh/tugsbayasgalan/78/orig 2025-12-04T09:21:04.4247465Z * [new branch] gh/tugsbayasgalan/79/base -> origin/gh/tugsbayasgalan/79/base 2025-12-04T09:21:04.4248672Z * [new branch] gh/tugsbayasgalan/79/head -> origin/gh/tugsbayasgalan/79/head 2025-12-04T09:21:04.4249880Z * [new branch] gh/tugsbayasgalan/79/orig -> origin/gh/tugsbayasgalan/79/orig 2025-12-04T09:21:04.4251513Z * [new branch] gh/tugsbayasgalan/8/base -> origin/gh/tugsbayasgalan/8/base 2025-12-04T09:21:04.4252619Z * [new branch] gh/tugsbayasgalan/8/head -> origin/gh/tugsbayasgalan/8/head 2025-12-04T09:21:04.4253851Z * [new branch] gh/tugsbayasgalan/8/orig -> origin/gh/tugsbayasgalan/8/orig 2025-12-04T09:21:04.4255497Z * [new branch] gh/tugsbayasgalan/80/base -> origin/gh/tugsbayasgalan/80/base 2025-12-04T09:21:04.4256599Z * [new branch] gh/tugsbayasgalan/80/head -> origin/gh/tugsbayasgalan/80/head 2025-12-04T09:21:04.4257775Z * [new branch] gh/tugsbayasgalan/80/orig -> origin/gh/tugsbayasgalan/80/orig 2025-12-04T09:21:04.4259482Z * [new branch] gh/tugsbayasgalan/81/base -> origin/gh/tugsbayasgalan/81/base 2025-12-04T09:21:04.4260567Z * [new branch] gh/tugsbayasgalan/81/head -> origin/gh/tugsbayasgalan/81/head 2025-12-04T09:21:04.4261750Z * [new branch] gh/tugsbayasgalan/81/orig -> origin/gh/tugsbayasgalan/81/orig 2025-12-04T09:21:04.4264374Z * [new branch] gh/tugsbayasgalan/82/base -> origin/gh/tugsbayasgalan/82/base 2025-12-04T09:21:04.4265758Z * [new branch] gh/tugsbayasgalan/82/head -> origin/gh/tugsbayasgalan/82/head 2025-12-04T09:21:04.4267359Z * [new branch] gh/tugsbayasgalan/82/orig -> origin/gh/tugsbayasgalan/82/orig 2025-12-04T09:21:04.4268923Z * [new branch] gh/tugsbayasgalan/83/base -> origin/gh/tugsbayasgalan/83/base 2025-12-04T09:21:04.4270234Z * [new branch] gh/tugsbayasgalan/83/head -> origin/gh/tugsbayasgalan/83/head 2025-12-04T09:21:04.4271451Z * [new branch] gh/tugsbayasgalan/83/orig -> origin/gh/tugsbayasgalan/83/orig 2025-12-04T09:21:04.4273132Z * [new branch] gh/tugsbayasgalan/84/base -> origin/gh/tugsbayasgalan/84/base 2025-12-04T09:21:04.4274436Z * [new branch] gh/tugsbayasgalan/84/head -> origin/gh/tugsbayasgalan/84/head 2025-12-04T09:21:04.4280171Z * [new branch] gh/tugsbayasgalan/84/orig -> origin/gh/tugsbayasgalan/84/orig 2025-12-04T09:21:04.4280475Z * [new branch] gh/tugsbayasgalan/85/base -> origin/gh/tugsbayasgalan/85/base 2025-12-04T09:21:04.4280781Z * [new branch] gh/tugsbayasgalan/85/head -> origin/gh/tugsbayasgalan/85/head 2025-12-04T09:21:04.4281092Z * [new branch] gh/tugsbayasgalan/85/orig -> origin/gh/tugsbayasgalan/85/orig 2025-12-04T09:21:04.4281474Z * [new branch] gh/tugsbayasgalan/86/base -> origin/gh/tugsbayasgalan/86/base 2025-12-04T09:21:04.4282851Z * [new branch] gh/tugsbayasgalan/86/head -> origin/gh/tugsbayasgalan/86/head 2025-12-04T09:21:04.4284099Z * [new branch] gh/tugsbayasgalan/86/orig -> origin/gh/tugsbayasgalan/86/orig 2025-12-04T09:21:04.4286117Z * [new branch] gh/tugsbayasgalan/87/base -> origin/gh/tugsbayasgalan/87/base 2025-12-04T09:21:04.4287479Z * [new branch] gh/tugsbayasgalan/87/head -> origin/gh/tugsbayasgalan/87/head 2025-12-04T09:21:04.4288721Z * [new branch] gh/tugsbayasgalan/87/orig -> origin/gh/tugsbayasgalan/87/orig 2025-12-04T09:21:04.4290616Z * [new branch] gh/tugsbayasgalan/88/base -> origin/gh/tugsbayasgalan/88/base 2025-12-04T09:21:04.4291770Z * [new branch] gh/tugsbayasgalan/88/head -> origin/gh/tugsbayasgalan/88/head 2025-12-04T09:21:04.4293050Z * [new branch] gh/tugsbayasgalan/88/orig -> origin/gh/tugsbayasgalan/88/orig 2025-12-04T09:21:04.4294703Z * [new branch] gh/tugsbayasgalan/89/base -> origin/gh/tugsbayasgalan/89/base 2025-12-04T09:21:04.4295796Z * [new branch] gh/tugsbayasgalan/89/head -> origin/gh/tugsbayasgalan/89/head 2025-12-04T09:21:04.4296942Z * [new branch] gh/tugsbayasgalan/89/orig -> origin/gh/tugsbayasgalan/89/orig 2025-12-04T09:21:04.4298582Z * [new branch] gh/tugsbayasgalan/9/base -> origin/gh/tugsbayasgalan/9/base 2025-12-04T09:21:04.4299686Z * [new branch] gh/tugsbayasgalan/9/head -> origin/gh/tugsbayasgalan/9/head 2025-12-04T09:21:04.4301391Z * [new branch] gh/tugsbayasgalan/9/orig -> origin/gh/tugsbayasgalan/9/orig 2025-12-04T09:21:04.4303708Z * [new branch] gh/tugsbayasgalan/90/base -> origin/gh/tugsbayasgalan/90/base 2025-12-04T09:21:04.4304853Z * [new branch] gh/tugsbayasgalan/90/head -> origin/gh/tugsbayasgalan/90/head 2025-12-04T09:21:04.4306022Z * [new branch] gh/tugsbayasgalan/90/orig -> origin/gh/tugsbayasgalan/90/orig 2025-12-04T09:21:04.4308117Z * [new branch] gh/tugsbayasgalan/91/base -> origin/gh/tugsbayasgalan/91/base 2025-12-04T09:21:04.4309289Z * [new branch] gh/tugsbayasgalan/91/head -> origin/gh/tugsbayasgalan/91/head 2025-12-04T09:21:04.4310506Z * [new branch] gh/tugsbayasgalan/91/orig -> origin/gh/tugsbayasgalan/91/orig 2025-12-04T09:21:04.4312382Z * [new branch] gh/tugsbayasgalan/92/base -> origin/gh/tugsbayasgalan/92/base 2025-12-04T09:21:04.4313603Z * [new branch] gh/tugsbayasgalan/92/head -> origin/gh/tugsbayasgalan/92/head 2025-12-04T09:21:04.4314812Z * [new branch] gh/tugsbayasgalan/92/orig -> origin/gh/tugsbayasgalan/92/orig 2025-12-04T09:21:04.4316651Z * [new branch] gh/tugsbayasgalan/93/base -> origin/gh/tugsbayasgalan/93/base 2025-12-04T09:21:04.4318014Z * [new branch] gh/tugsbayasgalan/93/head -> origin/gh/tugsbayasgalan/93/head 2025-12-04T09:21:04.4319299Z * [new branch] gh/tugsbayasgalan/93/orig -> origin/gh/tugsbayasgalan/93/orig 2025-12-04T09:21:04.4321253Z * [new branch] gh/v0i0/14/base -> origin/gh/v0i0/14/base 2025-12-04T09:21:04.4322370Z * [new branch] gh/v0i0/14/head -> origin/gh/v0i0/14/head 2025-12-04T09:21:04.4323573Z * [new branch] gh/v0i0/14/orig -> origin/gh/v0i0/14/orig 2025-12-04T09:21:04.4325080Z * [new branch] gh/v0i0/15/base -> origin/gh/v0i0/15/base 2025-12-04T09:21:04.4326818Z * [new branch] gh/v0i0/15/head -> origin/gh/v0i0/15/head 2025-12-04T09:21:04.4328045Z * [new branch] gh/v0i0/15/orig -> origin/gh/v0i0/15/orig 2025-12-04T09:21:04.4329665Z * [new branch] gh/v0i0/16/base -> origin/gh/v0i0/16/base 2025-12-04T09:21:04.4330905Z * [new branch] gh/v0i0/16/head -> origin/gh/v0i0/16/head 2025-12-04T09:21:04.4332111Z * [new branch] gh/v0i0/16/orig -> origin/gh/v0i0/16/orig 2025-12-04T09:21:04.4333692Z * [new branch] gh/v0i0/17/base -> origin/gh/v0i0/17/base 2025-12-04T09:21:04.4334894Z * [new branch] gh/v0i0/17/head -> origin/gh/v0i0/17/head 2025-12-04T09:21:04.4336061Z * [new branch] gh/v0i0/17/orig -> origin/gh/v0i0/17/orig 2025-12-04T09:21:04.4343062Z * [new branch] gh/v0i0/18/base -> origin/gh/v0i0/18/base 2025-12-04T09:21:04.4344383Z * [new branch] gh/v0i0/18/head -> origin/gh/v0i0/18/head 2025-12-04T09:21:04.4345648Z * [new branch] gh/v0i0/18/orig -> origin/gh/v0i0/18/orig 2025-12-04T09:21:04.4347798Z * [new branch] gh/v0i0/19/base -> origin/gh/v0i0/19/base 2025-12-04T09:21:04.4349094Z * [new branch] gh/v0i0/19/head -> origin/gh/v0i0/19/head 2025-12-04T09:21:04.4350298Z * [new branch] gh/v0i0/19/orig -> origin/gh/v0i0/19/orig 2025-12-04T09:21:04.4352380Z * [new branch] gh/vishal9-team/1/base -> origin/gh/vishal9-team/1/base 2025-12-04T09:21:04.4353659Z * [new branch] gh/vishal9-team/1/head -> origin/gh/vishal9-team/1/head 2025-12-04T09:21:04.4355146Z * [new branch] gh/vishal9-team/2/base -> origin/gh/vishal9-team/2/base 2025-12-04T09:21:04.4356366Z * [new branch] gh/vishal9-team/2/head -> origin/gh/vishal9-team/2/head 2025-12-04T09:21:04.4357578Z * [new branch] gh/vishal9-team/2/orig -> origin/gh/vishal9-team/2/orig 2025-12-04T09:21:04.4359374Z * [new branch] gh/vishal9-team/3/base -> origin/gh/vishal9-team/3/base 2025-12-04T09:21:04.4360598Z * [new branch] gh/vishal9-team/3/head -> origin/gh/vishal9-team/3/head 2025-12-04T09:21:04.4361801Z * [new branch] gh/vishal9-team/3/orig -> origin/gh/vishal9-team/3/orig 2025-12-04T09:21:04.4363358Z * [new branch] gh/vishal9-team/4/base -> origin/gh/vishal9-team/4/base 2025-12-04T09:21:04.4364535Z * [new branch] gh/vishal9-team/4/head -> origin/gh/vishal9-team/4/head 2025-12-04T09:21:04.4365768Z * [new branch] gh/vishal9-team/4/orig -> origin/gh/vishal9-team/4/orig 2025-12-04T09:21:04.4367594Z * [new branch] gh/vkuzo/1/next -> origin/gh/vkuzo/1/next 2025-12-04T09:21:04.4369146Z * [new branch] gh/vkuzo/2/next -> origin/gh/vkuzo/2/next 2025-12-04T09:21:04.4371143Z * [new branch] gh/vkuzo/3/next -> origin/gh/vkuzo/3/next 2025-12-04T09:21:04.4373137Z * [new branch] gh/wconstab/424/base -> origin/gh/wconstab/424/base 2025-12-04T09:21:04.4374425Z * [new branch] gh/wconstab/424/head -> origin/gh/wconstab/424/head 2025-12-04T09:21:04.4376209Z * [new branch] gh/wconstab/424/orig -> origin/gh/wconstab/424/orig 2025-12-04T09:21:04.4377946Z * [new branch] gh/wconstab/435/base -> origin/gh/wconstab/435/base 2025-12-04T09:21:04.4379229Z * [new branch] gh/wconstab/435/head -> origin/gh/wconstab/435/head 2025-12-04T09:21:04.4380450Z * [new branch] gh/wconstab/435/orig -> origin/gh/wconstab/435/orig 2025-12-04T09:21:04.4382013Z * [new branch] gh/wconstab/444/base -> origin/gh/wconstab/444/base 2025-12-04T09:21:04.4383220Z * [new branch] gh/wconstab/444/head -> origin/gh/wconstab/444/head 2025-12-04T09:21:04.4384429Z * [new branch] gh/wconstab/444/orig -> origin/gh/wconstab/444/orig 2025-12-04T09:21:04.4386059Z * [new branch] gh/wconstab/447/base -> origin/gh/wconstab/447/base 2025-12-04T09:21:04.4387648Z * [new branch] gh/wconstab/447/head -> origin/gh/wconstab/447/head 2025-12-04T09:21:04.4388856Z * [new branch] gh/wconstab/447/orig -> origin/gh/wconstab/447/orig 2025-12-04T09:21:04.4390577Z * [new branch] gh/wconstab/448/base -> origin/gh/wconstab/448/base 2025-12-04T09:21:04.4391842Z * [new branch] gh/wconstab/448/head -> origin/gh/wconstab/448/head 2025-12-04T09:21:04.4393072Z * [new branch] gh/wconstab/448/orig -> origin/gh/wconstab/448/orig 2025-12-04T09:21:04.4394692Z * [new branch] gh/wconstab/449/base -> origin/gh/wconstab/449/base 2025-12-04T09:21:04.4395911Z * [new branch] gh/wconstab/449/head -> origin/gh/wconstab/449/head 2025-12-04T09:21:04.4397225Z * [new branch] gh/wconstab/449/orig -> origin/gh/wconstab/449/orig 2025-12-04T09:21:04.4398683Z * [new branch] gh/wconstab/450/base -> origin/gh/wconstab/450/base 2025-12-04T09:21:04.4405422Z * [new branch] gh/wconstab/450/head -> origin/gh/wconstab/450/head 2025-12-04T09:21:04.4406838Z * [new branch] gh/wconstab/450/orig -> origin/gh/wconstab/450/orig 2025-12-04T09:21:04.4408356Z * [new branch] gh/wconstab/451/base -> origin/gh/wconstab/451/base 2025-12-04T09:21:04.4409724Z * [new branch] gh/wconstab/451/head -> origin/gh/wconstab/451/head 2025-12-04T09:21:04.4410948Z * [new branch] gh/wconstab/451/orig -> origin/gh/wconstab/451/orig 2025-12-04T09:21:04.4412672Z * [new branch] gh/wconstab/452/base -> origin/gh/wconstab/452/base 2025-12-04T09:21:04.4413957Z * [new branch] gh/wconstab/452/head -> origin/gh/wconstab/452/head 2025-12-04T09:21:04.4415190Z * [new branch] gh/wconstab/452/orig -> origin/gh/wconstab/452/orig 2025-12-04T09:21:04.4416678Z * [new branch] gh/wconstab/453/base -> origin/gh/wconstab/453/base 2025-12-04T09:21:04.4417988Z * [new branch] gh/wconstab/453/head -> origin/gh/wconstab/453/head 2025-12-04T09:21:04.4419231Z * [new branch] gh/wconstab/453/orig -> origin/gh/wconstab/453/orig 2025-12-04T09:21:04.4420666Z * [new branch] gh/wconstab/454/base -> origin/gh/wconstab/454/base 2025-12-04T09:21:04.4421907Z * [new branch] gh/wconstab/454/head -> origin/gh/wconstab/454/head 2025-12-04T09:21:04.4423092Z * [new branch] gh/wconstab/454/orig -> origin/gh/wconstab/454/orig 2025-12-04T09:21:04.4424713Z * [new branch] gh/wconstab/455/base -> origin/gh/wconstab/455/base 2025-12-04T09:21:04.4425951Z * [new branch] gh/wconstab/455/head -> origin/gh/wconstab/455/head 2025-12-04T09:21:04.4427474Z * [new branch] gh/wconstab/455/orig -> origin/gh/wconstab/455/orig 2025-12-04T09:21:04.4429413Z * [new branch] gh/wconstab/456/base -> origin/gh/wconstab/456/base 2025-12-04T09:21:04.4430988Z * [new branch] gh/wconstab/456/head -> origin/gh/wconstab/456/head 2025-12-04T09:21:04.4432374Z * [new branch] gh/wconstab/456/orig -> origin/gh/wconstab/456/orig 2025-12-04T09:21:04.4434179Z * [new branch] gh/wconstab/457/base -> origin/gh/wconstab/457/base 2025-12-04T09:21:04.4435430Z * [new branch] gh/wconstab/457/head -> origin/gh/wconstab/457/head 2025-12-04T09:21:04.4436675Z * [new branch] gh/wconstab/457/orig -> origin/gh/wconstab/457/orig 2025-12-04T09:21:04.4438353Z * [new branch] gh/wconstab/458/base -> origin/gh/wconstab/458/base 2025-12-04T09:21:04.4439740Z * [new branch] gh/wconstab/458/head -> origin/gh/wconstab/458/head 2025-12-04T09:21:04.4440919Z * [new branch] gh/wconstab/458/orig -> origin/gh/wconstab/458/orig 2025-12-04T09:21:04.4452427Z * [new branch] gh/wconstab/459/base -> origin/gh/wconstab/459/base 2025-12-04T09:21:04.4452845Z * [new branch] gh/wconstab/459/head -> origin/gh/wconstab/459/head 2025-12-04T09:21:04.4453216Z * [new branch] gh/wconstab/459/orig -> origin/gh/wconstab/459/orig 2025-12-04T09:21:04.4453487Z * [new branch] gh/wconstab/460/base -> origin/gh/wconstab/460/base 2025-12-04T09:21:04.4453774Z * [new branch] gh/wconstab/460/head -> origin/gh/wconstab/460/head 2025-12-04T09:21:04.4454027Z * [new branch] gh/wconstab/460/orig -> origin/gh/wconstab/460/orig 2025-12-04T09:21:04.4454288Z * [new branch] gh/wconstab/461/base -> origin/gh/wconstab/461/base 2025-12-04T09:21:04.4454549Z * [new branch] gh/wconstab/461/head -> origin/gh/wconstab/461/head 2025-12-04T09:21:04.4454801Z * [new branch] gh/wconstab/461/orig -> origin/gh/wconstab/461/orig 2025-12-04T09:21:04.4455595Z * [new branch] gh/wconstab/462/base -> origin/gh/wconstab/462/base 2025-12-04T09:21:04.4456904Z * [new branch] gh/wconstab/462/head -> origin/gh/wconstab/462/head 2025-12-04T09:21:04.4458167Z * [new branch] gh/wconstab/462/orig -> origin/gh/wconstab/462/orig 2025-12-04T09:21:04.4460208Z * [new branch] gh/wconstab/463/base -> origin/gh/wconstab/463/base 2025-12-04T09:21:04.4461531Z * [new branch] gh/wconstab/463/head -> origin/gh/wconstab/463/head 2025-12-04T09:21:04.4467913Z * [new branch] gh/wconstab/463/orig -> origin/gh/wconstab/463/orig 2025-12-04T09:21:04.4468180Z * [new branch] gh/wconstab/464/base -> origin/gh/wconstab/464/base 2025-12-04T09:21:04.4468540Z * [new branch] gh/wconstab/464/head -> origin/gh/wconstab/464/head 2025-12-04T09:21:04.4468803Z * [new branch] gh/wconstab/464/orig -> origin/gh/wconstab/464/orig 2025-12-04T09:21:04.4469433Z * [new branch] gh/wconstab/465/base -> origin/gh/wconstab/465/base 2025-12-04T09:21:04.4470866Z * [new branch] gh/wconstab/465/head -> origin/gh/wconstab/465/head 2025-12-04T09:21:04.4472016Z * [new branch] gh/wconstab/465/orig -> origin/gh/wconstab/465/orig 2025-12-04T09:21:04.4473821Z * [new branch] gh/wconstab/466/base -> origin/gh/wconstab/466/base 2025-12-04T09:21:04.4474972Z * [new branch] gh/wconstab/466/head -> origin/gh/wconstab/466/head 2025-12-04T09:21:04.4476156Z * [new branch] gh/wconstab/466/orig -> origin/gh/wconstab/466/orig 2025-12-04T09:21:04.4478842Z * [new branch] gh/wconstab/467/base -> origin/gh/wconstab/467/base 2025-12-04T09:21:04.4480121Z * [new branch] gh/wconstab/467/head -> origin/gh/wconstab/467/head 2025-12-04T09:21:04.4481383Z * [new branch] gh/wconstab/467/orig -> origin/gh/wconstab/467/orig 2025-12-04T09:21:04.4482871Z * [new branch] gh/wconstab/468/base -> origin/gh/wconstab/468/base 2025-12-04T09:21:04.4484044Z * [new branch] gh/wconstab/468/head -> origin/gh/wconstab/468/head 2025-12-04T09:21:04.4485192Z * [new branch] gh/wconstab/468/orig -> origin/gh/wconstab/468/orig 2025-12-04T09:21:04.4487389Z * [new branch] gh/weifengpy/39/base -> origin/gh/weifengpy/39/base 2025-12-04T09:21:04.4488529Z * [new branch] gh/weifengpy/39/head -> origin/gh/weifengpy/39/head 2025-12-04T09:21:04.4489767Z * [new branch] gh/weifengpy/39/orig -> origin/gh/weifengpy/39/orig 2025-12-04T09:21:04.4491463Z * [new branch] gh/weifengpy/40/base -> origin/gh/weifengpy/40/base 2025-12-04T09:21:04.4492675Z * [new branch] gh/weifengpy/40/head -> origin/gh/weifengpy/40/head 2025-12-04T09:21:04.4493921Z * [new branch] gh/weifengpy/40/orig -> origin/gh/weifengpy/40/orig 2025-12-04T09:21:04.4495673Z * [new branch] gh/weifengpy/41/base -> origin/gh/weifengpy/41/base 2025-12-04T09:21:04.4496951Z * [new branch] gh/weifengpy/41/head -> origin/gh/weifengpy/41/head 2025-12-04T09:21:04.4498269Z * [new branch] gh/weifengpy/41/orig -> origin/gh/weifengpy/41/orig 2025-12-04T09:21:04.4501073Z * [new branch] gh/williamwen42/250/base -> origin/gh/williamwen42/250/base 2025-12-04T09:21:04.4502877Z * [new branch] gh/williamwen42/250/head -> origin/gh/williamwen42/250/head 2025-12-04T09:21:04.4504091Z * [new branch] gh/williamwen42/250/orig -> origin/gh/williamwen42/250/orig 2025-12-04T09:21:04.4505847Z * [new branch] gh/williamwen42/279/base -> origin/gh/williamwen42/279/base 2025-12-04T09:21:04.4507442Z * [new branch] gh/williamwen42/279/head -> origin/gh/williamwen42/279/head 2025-12-04T09:21:04.4508710Z * [new branch] gh/williamwen42/279/orig -> origin/gh/williamwen42/279/orig 2025-12-04T09:21:04.4510419Z * [new branch] gh/williamwen42/282/base -> origin/gh/williamwen42/282/base 2025-12-04T09:21:04.4511651Z * [new branch] gh/williamwen42/282/head -> origin/gh/williamwen42/282/head 2025-12-04T09:21:04.4512912Z * [new branch] gh/williamwen42/282/orig -> origin/gh/williamwen42/282/orig 2025-12-04T09:21:04.4514729Z * [new branch] gh/williamwen42/287/base -> origin/gh/williamwen42/287/base 2025-12-04T09:21:04.4515951Z * [new branch] gh/williamwen42/287/head -> origin/gh/williamwen42/287/head 2025-12-04T09:21:04.4517217Z * [new branch] gh/williamwen42/287/orig -> origin/gh/williamwen42/287/orig 2025-12-04T09:21:04.4519545Z * [new branch] gh/williamwen42/288/base -> origin/gh/williamwen42/288/base 2025-12-04T09:21:04.4520790Z * [new branch] gh/williamwen42/288/head -> origin/gh/williamwen42/288/head 2025-12-04T09:21:04.4522026Z * [new branch] gh/williamwen42/288/orig -> origin/gh/williamwen42/288/orig 2025-12-04T09:21:04.4523789Z * [new branch] gh/williamwen42/296/base -> origin/gh/williamwen42/296/base 2025-12-04T09:21:04.4529946Z * [new branch] gh/williamwen42/296/head -> origin/gh/williamwen42/296/head 2025-12-04T09:21:04.4530305Z * [new branch] gh/williamwen42/296/orig -> origin/gh/williamwen42/296/orig 2025-12-04T09:21:04.4530638Z * [new branch] gh/williamwen42/297/base -> origin/gh/williamwen42/297/base 2025-12-04T09:21:04.4530981Z * [new branch] gh/williamwen42/297/head -> origin/gh/williamwen42/297/head 2025-12-04T09:21:04.4531286Z * [new branch] gh/williamwen42/297/orig -> origin/gh/williamwen42/297/orig 2025-12-04T09:21:04.4533195Z * [new branch] gh/williamwen42/306/base -> origin/gh/williamwen42/306/base 2025-12-04T09:21:04.4534141Z * [new branch] gh/williamwen42/306/head -> origin/gh/williamwen42/306/head 2025-12-04T09:21:04.4535456Z * [new branch] gh/williamwen42/306/orig -> origin/gh/williamwen42/306/orig 2025-12-04T09:21:04.4537263Z * [new branch] gh/williamwen42/309/base -> origin/gh/williamwen42/309/base 2025-12-04T09:21:04.4538698Z * [new branch] gh/williamwen42/309/head -> origin/gh/williamwen42/309/head 2025-12-04T09:21:04.4539862Z * [new branch] gh/williamwen42/309/orig -> origin/gh/williamwen42/309/orig 2025-12-04T09:21:04.4541437Z * [new branch] gh/williamwen42/310/base -> origin/gh/williamwen42/310/base 2025-12-04T09:21:04.4542634Z * [new branch] gh/williamwen42/310/head -> origin/gh/williamwen42/310/head 2025-12-04T09:21:04.4543936Z * [new branch] gh/williamwen42/310/orig -> origin/gh/williamwen42/310/orig 2025-12-04T09:21:04.4547105Z * [new branch] gh/williamwen42/311/base -> origin/gh/williamwen42/311/base 2025-12-04T09:21:04.4548529Z * [new branch] gh/williamwen42/311/head -> origin/gh/williamwen42/311/head 2025-12-04T09:21:04.4549825Z * [new branch] gh/williamwen42/311/orig -> origin/gh/williamwen42/311/orig 2025-12-04T09:21:04.4551337Z * [new branch] gh/williamwen42/319/base -> origin/gh/williamwen42/319/base 2025-12-04T09:21:04.4552568Z * [new branch] gh/williamwen42/319/head -> origin/gh/williamwen42/319/head 2025-12-04T09:21:04.4553806Z * [new branch] gh/williamwen42/319/orig -> origin/gh/williamwen42/319/orig 2025-12-04T09:21:04.4555512Z * [new branch] gh/williamwen42/325/base -> origin/gh/williamwen42/325/base 2025-12-04T09:21:04.4556949Z * [new branch] gh/williamwen42/325/head -> origin/gh/williamwen42/325/head 2025-12-04T09:21:04.4558709Z * [new branch] gh/williamwen42/325/orig -> origin/gh/williamwen42/325/orig 2025-12-04T09:21:04.4560473Z * [new branch] gh/williamwen42/326/base -> origin/gh/williamwen42/326/base 2025-12-04T09:21:04.4561741Z * [new branch] gh/williamwen42/326/head -> origin/gh/williamwen42/326/head 2025-12-04T09:21:04.4562999Z * [new branch] gh/williamwen42/326/orig -> origin/gh/williamwen42/326/orig 2025-12-04T09:21:04.4565116Z * [new branch] gh/williamwen42/327/base -> origin/gh/williamwen42/327/base 2025-12-04T09:21:04.4566344Z * [new branch] gh/williamwen42/327/head -> origin/gh/williamwen42/327/head 2025-12-04T09:21:04.4567532Z * [new branch] gh/williamwen42/327/orig -> origin/gh/williamwen42/327/orig 2025-12-04T09:21:04.4569236Z * [new branch] gh/williamwen42/328/base -> origin/gh/williamwen42/328/base 2025-12-04T09:21:04.4570865Z * [new branch] gh/williamwen42/328/head -> origin/gh/williamwen42/328/head 2025-12-04T09:21:04.4572013Z * [new branch] gh/williamwen42/328/orig -> origin/gh/williamwen42/328/orig 2025-12-04T09:21:04.4574065Z * [new branch] gh/williamwen42/329/base -> origin/gh/williamwen42/329/base 2025-12-04T09:21:04.4575487Z * [new branch] gh/williamwen42/329/head -> origin/gh/williamwen42/329/head 2025-12-04T09:21:04.4576723Z * [new branch] gh/williamwen42/329/orig -> origin/gh/williamwen42/329/orig 2025-12-04T09:21:04.4578372Z * [new branch] gh/williamwen42/330/base -> origin/gh/williamwen42/330/base 2025-12-04T09:21:04.4579679Z * [new branch] gh/williamwen42/330/head -> origin/gh/williamwen42/330/head 2025-12-04T09:21:04.4580911Z * [new branch] gh/williamwen42/330/orig -> origin/gh/williamwen42/330/orig 2025-12-04T09:21:04.4582466Z * [new branch] gh/williamwen42/331/base -> origin/gh/williamwen42/331/base 2025-12-04T09:21:04.4583649Z * [new branch] gh/williamwen42/331/head -> origin/gh/williamwen42/331/head 2025-12-04T09:21:04.4584886Z * [new branch] gh/williamwen42/331/orig -> origin/gh/williamwen42/331/orig 2025-12-04T09:21:04.4586353Z * [new branch] gh/williamwen42/332/base -> origin/gh/williamwen42/332/base 2025-12-04T09:21:04.4592924Z * [new branch] gh/williamwen42/332/head -> origin/gh/williamwen42/332/head 2025-12-04T09:21:04.4594172Z * [new branch] gh/williamwen42/332/orig -> origin/gh/williamwen42/332/orig 2025-12-04T09:21:04.4596582Z * [new branch] gh/williamwen42/333/base -> origin/gh/williamwen42/333/base 2025-12-04T09:21:04.4597959Z * [new branch] gh/williamwen42/333/head -> origin/gh/williamwen42/333/head 2025-12-04T09:21:04.4599272Z * [new branch] gh/williamwen42/333/orig -> origin/gh/williamwen42/333/orig 2025-12-04T09:21:04.4601411Z * [new branch] gh/williamwen42/334/base -> origin/gh/williamwen42/334/base 2025-12-04T09:21:04.4602925Z * [new branch] gh/williamwen42/334/head -> origin/gh/williamwen42/334/head 2025-12-04T09:21:04.4604213Z * [new branch] gh/williamwen42/334/orig -> origin/gh/williamwen42/334/orig 2025-12-04T09:21:04.4609803Z * [new branch] gh/williamwen42/335/base -> origin/gh/williamwen42/335/base 2025-12-04T09:21:04.4611205Z * [new branch] gh/williamwen42/335/head -> origin/gh/williamwen42/335/head 2025-12-04T09:21:04.4613055Z * [new branch] gh/williamwen42/335/orig -> origin/gh/williamwen42/335/orig 2025-12-04T09:21:04.4614782Z * [new branch] gh/williamwen42/336/base -> origin/gh/williamwen42/336/base 2025-12-04T09:21:04.4615939Z * [new branch] gh/williamwen42/336/head -> origin/gh/williamwen42/336/head 2025-12-04T09:21:04.4617545Z * [new branch] gh/williamwen42/336/orig -> origin/gh/williamwen42/336/orig 2025-12-04T09:21:04.4619295Z * [new branch] gh/williamwen42/337/base -> origin/gh/williamwen42/337/base 2025-12-04T09:21:04.4620569Z * [new branch] gh/williamwen42/337/head -> origin/gh/williamwen42/337/head 2025-12-04T09:21:04.4621814Z * [new branch] gh/williamwen42/337/orig -> origin/gh/williamwen42/337/orig 2025-12-04T09:21:04.4623580Z * [new branch] gh/williamwen42/338/base -> origin/gh/williamwen42/338/base 2025-12-04T09:21:04.4624804Z * [new branch] gh/williamwen42/338/head -> origin/gh/williamwen42/338/head 2025-12-04T09:21:04.4626015Z * [new branch] gh/williamwen42/338/orig -> origin/gh/williamwen42/338/orig 2025-12-04T09:21:04.4628093Z * [new branch] gh/williamwen42/339/base -> origin/gh/williamwen42/339/base 2025-12-04T09:21:04.4629383Z * [new branch] gh/williamwen42/339/head -> origin/gh/williamwen42/339/head 2025-12-04T09:21:04.4630492Z * [new branch] gh/williamwen42/339/orig -> origin/gh/williamwen42/339/orig 2025-12-04T09:21:04.4632337Z * [new branch] gh/williamwen42/340/base -> origin/gh/williamwen42/340/base 2025-12-04T09:21:04.4633497Z * [new branch] gh/williamwen42/340/head -> origin/gh/williamwen42/340/head 2025-12-04T09:21:04.4634671Z * [new branch] gh/williamwen42/340/orig -> origin/gh/williamwen42/340/orig 2025-12-04T09:21:04.4636433Z * [new branch] gh/williamwen42/341/base -> origin/gh/williamwen42/341/base 2025-12-04T09:21:04.4637807Z * [new branch] gh/williamwen42/341/head -> origin/gh/williamwen42/341/head 2025-12-04T09:21:04.4639147Z * [new branch] gh/williamwen42/341/orig -> origin/gh/williamwen42/341/orig 2025-12-04T09:21:04.4640841Z * [new branch] gh/williamwen42/342/base -> origin/gh/williamwen42/342/base 2025-12-04T09:21:04.4642095Z * [new branch] gh/williamwen42/342/head -> origin/gh/williamwen42/342/head 2025-12-04T09:21:04.4643297Z * [new branch] gh/williamwen42/342/orig -> origin/gh/williamwen42/342/orig 2025-12-04T09:21:04.4644968Z * [new branch] gh/williamwen42/343/base -> origin/gh/williamwen42/343/base 2025-12-04T09:21:04.4646272Z * [new branch] gh/williamwen42/343/head -> origin/gh/williamwen42/343/head 2025-12-04T09:21:04.4647429Z * [new branch] gh/williamwen42/343/orig -> origin/gh/williamwen42/343/orig 2025-12-04T09:21:04.4649115Z * [new branch] gh/williamwen42/344/base -> origin/gh/williamwen42/344/base 2025-12-04T09:21:04.4650923Z * [new branch] gh/williamwen42/344/head -> origin/gh/williamwen42/344/head 2025-12-04T09:21:04.4652083Z * [new branch] gh/williamwen42/344/orig -> origin/gh/williamwen42/344/orig 2025-12-04T09:21:04.4653846Z * [new branch] gh/williamwen42/345/base -> origin/gh/williamwen42/345/base 2025-12-04T09:21:04.4655348Z * [new branch] gh/williamwen42/345/head -> origin/gh/williamwen42/345/head 2025-12-04T09:21:04.4656675Z * [new branch] gh/williamwen42/345/orig -> origin/gh/williamwen42/345/orig 2025-12-04T09:21:04.4658358Z * [new branch] gh/williamwen42/346/base -> origin/gh/williamwen42/346/base 2025-12-04T09:21:04.4659651Z * [new branch] gh/williamwen42/346/head -> origin/gh/williamwen42/346/head 2025-12-04T09:21:04.4660916Z * [new branch] gh/williamwen42/346/orig -> origin/gh/williamwen42/346/orig 2025-12-04T09:21:04.4662731Z * [new branch] gh/williamwen42/347/base -> origin/gh/williamwen42/347/base 2025-12-04T09:21:04.4663897Z * [new branch] gh/williamwen42/347/head -> origin/gh/williamwen42/347/head 2025-12-04T09:21:04.4665108Z * [new branch] gh/williamwen42/347/orig -> origin/gh/williamwen42/347/orig 2025-12-04T09:21:04.4666871Z * [new branch] gh/williamwen42/348/base -> origin/gh/williamwen42/348/base 2025-12-04T09:21:04.4668118Z * [new branch] gh/williamwen42/348/head -> origin/gh/williamwen42/348/head 2025-12-04T09:21:04.4669305Z * [new branch] gh/williamwen42/348/orig -> origin/gh/williamwen42/348/orig 2025-12-04T09:21:04.4670782Z * [new branch] gh/williamwen42/349/base -> origin/gh/williamwen42/349/base 2025-12-04T09:21:04.4672130Z * [new branch] gh/williamwen42/349/head -> origin/gh/williamwen42/349/head 2025-12-04T09:21:04.4673393Z * [new branch] gh/williamwen42/349/orig -> origin/gh/williamwen42/349/orig 2025-12-04T09:21:04.4675157Z * [new branch] gh/williamwen42/350/base -> origin/gh/williamwen42/350/base 2025-12-04T09:21:04.4676373Z * [new branch] gh/williamwen42/350/head -> origin/gh/williamwen42/350/head 2025-12-04T09:21:04.4677645Z * [new branch] gh/williamwen42/350/orig -> origin/gh/williamwen42/350/orig 2025-12-04T09:21:04.4679494Z * [new branch] gh/williamwen42/351/base -> origin/gh/williamwen42/351/base 2025-12-04T09:21:04.4683170Z * [new branch] gh/williamwen42/351/head -> origin/gh/williamwen42/351/head 2025-12-04T09:21:04.4684864Z * [new branch] gh/williamwen42/351/orig -> origin/gh/williamwen42/351/orig 2025-12-04T09:21:04.4685142Z * [new branch] gh/williamwen42/352/base -> origin/gh/williamwen42/352/base 2025-12-04T09:21:04.4685423Z * [new branch] gh/williamwen42/352/head -> origin/gh/williamwen42/352/head 2025-12-04T09:21:04.4685911Z * [new branch] gh/williamwen42/352/orig -> origin/gh/williamwen42/352/orig 2025-12-04T09:21:04.4688078Z * [new branch] gh/williamwen42/353/base -> origin/gh/williamwen42/353/base 2025-12-04T09:21:04.4689495Z * [new branch] gh/williamwen42/353/head -> origin/gh/williamwen42/353/head 2025-12-04T09:21:04.4690720Z * [new branch] gh/williamwen42/353/orig -> origin/gh/williamwen42/353/orig 2025-12-04T09:21:04.4692300Z * [new branch] gh/williamwen42/354/base -> origin/gh/williamwen42/354/base 2025-12-04T09:21:04.4693595Z * [new branch] gh/williamwen42/354/head -> origin/gh/williamwen42/354/head 2025-12-04T09:21:04.4694802Z * [new branch] gh/williamwen42/354/orig -> origin/gh/williamwen42/354/orig 2025-12-04T09:21:04.4696426Z * [new branch] gh/williamwen42/355/base -> origin/gh/williamwen42/355/base 2025-12-04T09:21:04.4697579Z * [new branch] gh/williamwen42/355/head -> origin/gh/williamwen42/355/head 2025-12-04T09:21:04.4698758Z * [new branch] gh/williamwen42/355/orig -> origin/gh/williamwen42/355/orig 2025-12-04T09:21:04.4700475Z * [new branch] gh/williamwen42/356/base -> origin/gh/williamwen42/356/base 2025-12-04T09:21:04.4702536Z * [new branch] gh/williamwen42/356/head -> origin/gh/williamwen42/356/head 2025-12-04T09:21:04.4703696Z * [new branch] gh/williamwen42/356/orig -> origin/gh/williamwen42/356/orig 2025-12-04T09:21:04.4705483Z * [new branch] gh/williamwen42/357/base -> origin/gh/williamwen42/357/base 2025-12-04T09:21:04.4706940Z * [new branch] gh/williamwen42/357/head -> origin/gh/williamwen42/357/head 2025-12-04T09:21:04.4708284Z * [new branch] gh/williamwen42/357/orig -> origin/gh/williamwen42/357/orig 2025-12-04T09:21:04.4709971Z * [new branch] gh/williamwen42/358/base -> origin/gh/williamwen42/358/base 2025-12-04T09:21:04.4711165Z * [new branch] gh/williamwen42/358/head -> origin/gh/williamwen42/358/head 2025-12-04T09:21:04.4712802Z * [new branch] gh/williamwen42/358/orig -> origin/gh/williamwen42/358/orig 2025-12-04T09:21:04.4717595Z * [new branch] gh/xmfan/169/base -> origin/gh/xmfan/169/base 2025-12-04T09:21:04.4717867Z * [new branch] gh/xmfan/169/head -> origin/gh/xmfan/169/head 2025-12-04T09:21:04.4718112Z * [new branch] gh/xmfan/170/base -> origin/gh/xmfan/170/base 2025-12-04T09:21:04.4718695Z * [new branch] gh/xmfan/170/head -> origin/gh/xmfan/170/head 2025-12-04T09:21:04.4720348Z * [new branch] gh/xmfan/274/base -> origin/gh/xmfan/274/base 2025-12-04T09:21:04.4721622Z * [new branch] gh/xmfan/274/head -> origin/gh/xmfan/274/head 2025-12-04T09:21:04.4722943Z * [new branch] gh/xmfan/274/orig -> origin/gh/xmfan/274/orig 2025-12-04T09:21:04.4724589Z * [new branch] gh/xmfan/277/base -> origin/gh/xmfan/277/base 2025-12-04T09:21:04.4725787Z * [new branch] gh/xmfan/277/head -> origin/gh/xmfan/277/head 2025-12-04T09:21:04.4727091Z * [new branch] gh/xmfan/277/orig -> origin/gh/xmfan/277/orig 2025-12-04T09:21:04.4728730Z * [new branch] gh/xmfan/301/base -> origin/gh/xmfan/301/base 2025-12-04T09:21:04.4729782Z * [new branch] gh/xmfan/301/head -> origin/gh/xmfan/301/head 2025-12-04T09:21:04.4730918Z * [new branch] gh/xmfan/301/orig -> origin/gh/xmfan/301/orig 2025-12-04T09:21:04.4732509Z * [new branch] gh/xmfan/304/base -> origin/gh/xmfan/304/base 2025-12-04T09:21:04.4733668Z * [new branch] gh/xmfan/304/head -> origin/gh/xmfan/304/head 2025-12-04T09:21:04.4734884Z * [new branch] gh/xmfan/304/orig -> origin/gh/xmfan/304/orig 2025-12-04T09:21:04.4736405Z * [new branch] gh/xmfan/309/base -> origin/gh/xmfan/309/base 2025-12-04T09:21:04.4737572Z * [new branch] gh/xmfan/309/head -> origin/gh/xmfan/309/head 2025-12-04T09:21:04.4738835Z * [new branch] gh/xmfan/309/orig -> origin/gh/xmfan/309/orig 2025-12-04T09:21:04.4740443Z * [new branch] gh/xmfan/310/base -> origin/gh/xmfan/310/base 2025-12-04T09:21:04.4741577Z * [new branch] gh/xmfan/310/head -> origin/gh/xmfan/310/head 2025-12-04T09:21:04.4743180Z * [new branch] gh/xmfan/310/orig -> origin/gh/xmfan/310/orig 2025-12-04T09:21:04.4744763Z * [new branch] gh/xmfan/311/base -> origin/gh/xmfan/311/base 2025-12-04T09:21:04.4745954Z * [new branch] gh/xmfan/311/head -> origin/gh/xmfan/311/head 2025-12-04T09:21:04.4748011Z * [new branch] gh/xmfan/311/orig -> origin/gh/xmfan/311/orig 2025-12-04T09:21:04.4749636Z * [new branch] gh/xmfan/312/base -> origin/gh/xmfan/312/base 2025-12-04T09:21:04.4750860Z * [new branch] gh/xmfan/312/head -> origin/gh/xmfan/312/head 2025-12-04T09:21:04.4752102Z * [new branch] gh/xmfan/312/orig -> origin/gh/xmfan/312/orig 2025-12-04T09:21:04.4753686Z * [new branch] gh/xmfan/313/base -> origin/gh/xmfan/313/base 2025-12-04T09:21:04.4754894Z * [new branch] gh/xmfan/313/head -> origin/gh/xmfan/313/head 2025-12-04T09:21:04.4756213Z * [new branch] gh/xmfan/313/orig -> origin/gh/xmfan/313/orig 2025-12-04T09:21:04.4758232Z * [new branch] gh/xuanzhang816/27/base -> origin/gh/xuanzhang816/27/base 2025-12-04T09:21:04.4759525Z * [new branch] gh/xuanzhang816/27/head -> origin/gh/xuanzhang816/27/head 2025-12-04T09:21:04.4760731Z * [new branch] gh/xuanzhang816/27/orig -> origin/gh/xuanzhang816/27/orig 2025-12-04T09:21:04.4762422Z * [new branch] gh/xuanzhang816/32/base -> origin/gh/xuanzhang816/32/base 2025-12-04T09:21:04.4763588Z * [new branch] gh/xuanzhang816/32/head -> origin/gh/xuanzhang816/32/head 2025-12-04T09:21:04.4764775Z * [new branch] gh/xuanzhang816/32/orig -> origin/gh/xuanzhang816/32/orig 2025-12-04T09:21:04.4766414Z * [new branch] gh/xuanzhang816/33/base -> origin/gh/xuanzhang816/33/base 2025-12-04T09:21:04.4767577Z * [new branch] gh/xuanzhang816/33/head -> origin/gh/xuanzhang816/33/head 2025-12-04T09:21:04.4768860Z * [new branch] gh/xuanzhang816/33/orig -> origin/gh/xuanzhang816/33/orig 2025-12-04T09:21:04.4770792Z * [new branch] gh/xuanzhang816/34/base -> origin/gh/xuanzhang816/34/base 2025-12-04T09:21:04.4772136Z * [new branch] gh/xuanzhang816/34/head -> origin/gh/xuanzhang816/34/head 2025-12-04T09:21:04.4773411Z * [new branch] gh/xuanzhang816/34/orig -> origin/gh/xuanzhang816/34/orig 2025-12-04T09:21:04.4780939Z * [new branch] gh/xuanzhang816/35/base -> origin/gh/xuanzhang816/35/base 2025-12-04T09:21:04.4782390Z * [new branch] gh/xuanzhang816/35/head -> origin/gh/xuanzhang816/35/head 2025-12-04T09:21:04.4783384Z * [new branch] gh/xuanzhang816/35/orig -> origin/gh/xuanzhang816/35/orig 2025-12-04T09:21:04.4785415Z * [new branch] gh/yanbing-j/11/base -> origin/gh/yanbing-j/11/base 2025-12-04T09:21:04.4786720Z * [new branch] gh/yanbing-j/11/head -> origin/gh/yanbing-j/11/head 2025-12-04T09:21:04.4788055Z * [new branch] gh/yanbing-j/11/orig -> origin/gh/yanbing-j/11/orig 2025-12-04T09:21:04.4789667Z * [new branch] gh/yanbing-j/12/base -> origin/gh/yanbing-j/12/base 2025-12-04T09:21:04.4791369Z * [new branch] gh/yanbing-j/12/head -> origin/gh/yanbing-j/12/head 2025-12-04T09:21:04.4792589Z * [new branch] gh/yanbing-j/12/orig -> origin/gh/yanbing-j/12/orig 2025-12-04T09:21:04.4794353Z * [new branch] gh/yanbing-j/13/base -> origin/gh/yanbing-j/13/base 2025-12-04T09:21:04.4795591Z * [new branch] gh/yanbing-j/13/head -> origin/gh/yanbing-j/13/head 2025-12-04T09:21:04.4796789Z * [new branch] gh/yanbing-j/13/orig -> origin/gh/yanbing-j/13/orig 2025-12-04T09:21:04.4798779Z * [new branch] gh/yanbing-j/14/base -> origin/gh/yanbing-j/14/base 2025-12-04T09:21:04.4799992Z * [new branch] gh/yanbing-j/14/head -> origin/gh/yanbing-j/14/head 2025-12-04T09:21:04.4801808Z * [new branch] gh/yanbing-j/14/orig -> origin/gh/yanbing-j/14/orig 2025-12-04T09:21:04.4804632Z * [new branch] gh/yanbing-j/15/base -> origin/gh/yanbing-j/15/base 2025-12-04T09:21:04.4805903Z * [new branch] gh/yanbing-j/15/head -> origin/gh/yanbing-j/15/head 2025-12-04T09:21:04.4807143Z * [new branch] gh/yanbing-j/15/orig -> origin/gh/yanbing-j/15/orig 2025-12-04T09:21:04.4809106Z * [new branch] gh/yanbing-j/18/base -> origin/gh/yanbing-j/18/base 2025-12-04T09:21:04.4810391Z * [new branch] gh/yanbing-j/18/head -> origin/gh/yanbing-j/18/head 2025-12-04T09:21:04.4811600Z * [new branch] gh/yanbing-j/18/orig -> origin/gh/yanbing-j/18/orig 2025-12-04T09:21:04.4813387Z * [new branch] gh/yanbing-j/19/base -> origin/gh/yanbing-j/19/base 2025-12-04T09:21:04.4814691Z * [new branch] gh/yanbing-j/19/head -> origin/gh/yanbing-j/19/head 2025-12-04T09:21:04.4815897Z * [new branch] gh/yanbing-j/19/orig -> origin/gh/yanbing-j/19/orig 2025-12-04T09:21:04.4817460Z * [new branch] gh/yanbing-j/20/base -> origin/gh/yanbing-j/20/base 2025-12-04T09:21:04.4818605Z * [new branch] gh/yanbing-j/20/head -> origin/gh/yanbing-j/20/head 2025-12-04T09:21:04.4819831Z * [new branch] gh/yanbing-j/20/orig -> origin/gh/yanbing-j/20/orig 2025-12-04T09:21:04.4821442Z * [new branch] gh/yanbing-j/21/base -> origin/gh/yanbing-j/21/base 2025-12-04T09:21:04.4822663Z * [new branch] gh/yanbing-j/21/head -> origin/gh/yanbing-j/21/head 2025-12-04T09:21:04.4824198Z * [new branch] gh/yanbing-j/22/base -> origin/gh/yanbing-j/22/base 2025-12-04T09:21:04.4825426Z * [new branch] gh/yanbing-j/22/head -> origin/gh/yanbing-j/22/head 2025-12-04T09:21:04.4826772Z * [new branch] gh/yanbing-j/22/orig -> origin/gh/yanbing-j/22/orig 2025-12-04T09:21:04.4828598Z * [new branch] gh/yanbing-j/23/base -> origin/gh/yanbing-j/23/base 2025-12-04T09:21:04.4829950Z * [new branch] gh/yanbing-j/23/head -> origin/gh/yanbing-j/23/head 2025-12-04T09:21:04.4831628Z * [new branch] gh/yanbing-j/23/orig -> origin/gh/yanbing-j/23/orig 2025-12-04T09:21:04.4833717Z * [new branch] gh/yanbing-j/24/base -> origin/gh/yanbing-j/24/base 2025-12-04T09:21:04.4834913Z * [new branch] gh/yanbing-j/24/head -> origin/gh/yanbing-j/24/head 2025-12-04T09:21:04.4836202Z * [new branch] gh/yanbing-j/24/orig -> origin/gh/yanbing-j/24/orig 2025-12-04T09:21:04.4842713Z * [new branch] gh/yanbing-j/25/base -> origin/gh/yanbing-j/25/base 2025-12-04T09:21:04.4843951Z * [new branch] gh/yanbing-j/25/head -> origin/gh/yanbing-j/25/head 2025-12-04T09:21:04.4845186Z * [new branch] gh/yanbing-j/25/orig -> origin/gh/yanbing-j/25/orig 2025-12-04T09:21:04.4846809Z * [new branch] gh/yanbing-j/26/base -> origin/gh/yanbing-j/26/base 2025-12-04T09:21:04.4848445Z * [new branch] gh/yanbing-j/26/head -> origin/gh/yanbing-j/26/head 2025-12-04T09:21:04.4849879Z * [new branch] gh/yanbing-j/26/orig -> origin/gh/yanbing-j/26/orig 2025-12-04T09:21:04.4852049Z * [new branch] gh/yang-yu-hang/1/base -> origin/gh/yang-yu-hang/1/base 2025-12-04T09:21:04.4853499Z * [new branch] gh/yang-yu-hang/1/head -> origin/gh/yang-yu-hang/1/head 2025-12-04T09:21:04.4854865Z * [new branch] gh/yang-yu-hang/1/orig -> origin/gh/yang-yu-hang/1/orig 2025-12-04T09:21:04.4856512Z * [new branch] gh/yang-yu-hang/2/base -> origin/gh/yang-yu-hang/2/base 2025-12-04T09:21:04.4857975Z * [new branch] gh/yang-yu-hang/2/head -> origin/gh/yang-yu-hang/2/head 2025-12-04T09:21:04.4859460Z * [new branch] gh/yang-yu-hang/2/orig -> origin/gh/yang-yu-hang/2/orig 2025-12-04T09:21:04.4861156Z * [new branch] gh/yang-yu-hang/3/base -> origin/gh/yang-yu-hang/3/base 2025-12-04T09:21:04.4862394Z * [new branch] gh/yang-yu-hang/3/head -> origin/gh/yang-yu-hang/3/head 2025-12-04T09:21:04.4863687Z * [new branch] gh/yang-yu-hang/3/orig -> origin/gh/yang-yu-hang/3/orig 2025-12-04T09:21:04.4865985Z * [new branch] gh/yangw-dev/12/base -> origin/gh/yangw-dev/12/base 2025-12-04T09:21:04.4867573Z * [new branch] gh/yangw-dev/12/head -> origin/gh/yangw-dev/12/head 2025-12-04T09:21:04.4868867Z * [new branch] gh/yangw-dev/12/orig -> origin/gh/yangw-dev/12/orig 2025-12-04T09:21:04.4870620Z * [new branch] gh/yangw-dev/13/base -> origin/gh/yangw-dev/13/base 2025-12-04T09:21:04.4871905Z * [new branch] gh/yangw-dev/13/head -> origin/gh/yangw-dev/13/head 2025-12-04T09:21:04.4873105Z * [new branch] gh/yangw-dev/13/orig -> origin/gh/yangw-dev/13/orig 2025-12-04T09:21:04.4874746Z * [new branch] gh/yangw-dev/14/base -> origin/gh/yangw-dev/14/base 2025-12-04T09:21:04.4875952Z * [new branch] gh/yangw-dev/14/head -> origin/gh/yangw-dev/14/head 2025-12-04T09:21:04.4877173Z * [new branch] gh/yangw-dev/14/orig -> origin/gh/yangw-dev/14/orig 2025-12-04T09:21:04.4878844Z * [new branch] gh/yangw-dev/15/base -> origin/gh/yangw-dev/15/base 2025-12-04T09:21:04.4880182Z * [new branch] gh/yangw-dev/15/head -> origin/gh/yangw-dev/15/head 2025-12-04T09:21:04.4881368Z * [new branch] gh/yangw-dev/15/orig -> origin/gh/yangw-dev/15/orig 2025-12-04T09:21:04.4882908Z * [new branch] gh/yangw-dev/19/base -> origin/gh/yangw-dev/19/base 2025-12-04T09:21:04.4884105Z * [new branch] gh/yangw-dev/19/head -> origin/gh/yangw-dev/19/head 2025-12-04T09:21:04.4885282Z * [new branch] gh/yangw-dev/19/orig -> origin/gh/yangw-dev/19/orig 2025-12-04T09:21:04.4887029Z * [new branch] gh/yangw-dev/26/base -> origin/gh/yangw-dev/26/base 2025-12-04T09:21:04.4888190Z * [new branch] gh/yangw-dev/26/head -> origin/gh/yangw-dev/26/head 2025-12-04T09:21:04.4889350Z * [new branch] gh/yangw-dev/26/orig -> origin/gh/yangw-dev/26/orig 2025-12-04T09:21:04.4890891Z * [new branch] gh/yangw-dev/27/base -> origin/gh/yangw-dev/27/base 2025-12-04T09:21:04.4892110Z * [new branch] gh/yangw-dev/27/head -> origin/gh/yangw-dev/27/head 2025-12-04T09:21:04.4893238Z * [new branch] gh/yangw-dev/27/orig -> origin/gh/yangw-dev/27/orig 2025-12-04T09:21:04.4895219Z * [new branch] gh/ydwu4/292/base -> origin/gh/ydwu4/292/base 2025-12-04T09:21:04.4896389Z * [new branch] gh/ydwu4/292/head -> origin/gh/ydwu4/292/head 2025-12-04T09:21:04.4897564Z * [new branch] gh/ydwu4/292/orig -> origin/gh/ydwu4/292/orig 2025-12-04T09:21:04.4899264Z * [new branch] gh/ydwu4/294/base -> origin/gh/ydwu4/294/base 2025-12-04T09:21:04.4906023Z * [new branch] gh/ydwu4/294/head -> origin/gh/ydwu4/294/head 2025-12-04T09:21:04.4907571Z * [new branch] gh/ydwu4/294/orig -> origin/gh/ydwu4/294/orig 2025-12-04T09:21:04.4909464Z * [new branch] gh/ydwu4/295/base -> origin/gh/ydwu4/295/base 2025-12-04T09:21:04.4910765Z * [new branch] gh/ydwu4/295/head -> origin/gh/ydwu4/295/head 2025-12-04T09:21:04.4911990Z * [new branch] gh/ydwu4/295/orig -> origin/gh/ydwu4/295/orig 2025-12-04T09:21:04.4913541Z * [new branch] gh/ydwu4/296/base -> origin/gh/ydwu4/296/base 2025-12-04T09:21:04.4914690Z * [new branch] gh/ydwu4/296/head -> origin/gh/ydwu4/296/head 2025-12-04T09:21:04.4916037Z * [new branch] gh/ydwu4/296/orig -> origin/gh/ydwu4/296/orig 2025-12-04T09:21:04.4918019Z * [new branch] gh/ydwu4/306/base -> origin/gh/ydwu4/306/base 2025-12-04T09:21:04.4919374Z * [new branch] gh/ydwu4/306/head -> origin/gh/ydwu4/306/head 2025-12-04T09:21:04.4920615Z * [new branch] gh/ydwu4/306/orig -> origin/gh/ydwu4/306/orig 2025-12-04T09:21:04.4922216Z * [new branch] gh/ydwu4/312/base -> origin/gh/ydwu4/312/base 2025-12-04T09:21:04.4923424Z * [new branch] gh/ydwu4/312/head -> origin/gh/ydwu4/312/head 2025-12-04T09:21:04.4925217Z * [new branch] gh/ydwu4/312/orig -> origin/gh/ydwu4/312/orig 2025-12-04T09:21:04.4926806Z * [new branch] gh/ydwu4/322/base -> origin/gh/ydwu4/322/base 2025-12-04T09:21:04.4927964Z * [new branch] gh/ydwu4/322/head -> origin/gh/ydwu4/322/head 2025-12-04T09:21:04.4929169Z * [new branch] gh/ydwu4/322/orig -> origin/gh/ydwu4/322/orig 2025-12-04T09:21:04.4930689Z * [new branch] gh/ydwu4/327/base -> origin/gh/ydwu4/327/base 2025-12-04T09:21:04.4931922Z * [new branch] gh/ydwu4/327/head -> origin/gh/ydwu4/327/head 2025-12-04T09:21:04.4933147Z * [new branch] gh/ydwu4/327/orig -> origin/gh/ydwu4/327/orig 2025-12-04T09:21:04.4934828Z * [new branch] gh/ydwu4/328/base -> origin/gh/ydwu4/328/base 2025-12-04T09:21:04.4935946Z * [new branch] gh/ydwu4/328/head -> origin/gh/ydwu4/328/head 2025-12-04T09:21:04.4937044Z * [new branch] gh/ydwu4/328/orig -> origin/gh/ydwu4/328/orig 2025-12-04T09:21:04.4938471Z * [new branch] gh/ydwu4/329/base -> origin/gh/ydwu4/329/base 2025-12-04T09:21:04.4939701Z * [new branch] gh/ydwu4/329/head -> origin/gh/ydwu4/329/head 2025-12-04T09:21:04.4940963Z * [new branch] gh/ydwu4/329/orig -> origin/gh/ydwu4/329/orig 2025-12-04T09:21:04.4942728Z * [new branch] gh/ydwu4/330/base -> origin/gh/ydwu4/330/base 2025-12-04T09:21:04.4943799Z * [new branch] gh/ydwu4/330/head -> origin/gh/ydwu4/330/head 2025-12-04T09:21:04.4944977Z * [new branch] gh/ydwu4/330/orig -> origin/gh/ydwu4/330/orig 2025-12-04T09:21:04.4946392Z * [new branch] gh/ydwu4/331/base -> origin/gh/ydwu4/331/base 2025-12-04T09:21:04.4948155Z * [new branch] gh/ydwu4/331/head -> origin/gh/ydwu4/331/head 2025-12-04T09:21:04.4949265Z * [new branch] gh/ydwu4/331/orig -> origin/gh/ydwu4/331/orig 2025-12-04T09:21:04.4950708Z * [new branch] gh/ydwu4/332/base -> origin/gh/ydwu4/332/base 2025-12-04T09:21:04.4951925Z * [new branch] gh/ydwu4/332/head -> origin/gh/ydwu4/332/head 2025-12-04T09:21:04.4953118Z * [new branch] gh/ydwu4/332/orig -> origin/gh/ydwu4/332/orig 2025-12-04T09:21:04.4954643Z * [new branch] gh/ydwu4/333/base -> origin/gh/ydwu4/333/base 2025-12-04T09:21:04.4955943Z * [new branch] gh/ydwu4/333/head -> origin/gh/ydwu4/333/head 2025-12-04T09:21:04.4957289Z * [new branch] gh/ydwu4/333/orig -> origin/gh/ydwu4/333/orig 2025-12-04T09:21:04.4958930Z * [new branch] gh/ydwu4/334/base -> origin/gh/ydwu4/334/base 2025-12-04T09:21:04.4960104Z * [new branch] gh/ydwu4/334/head -> origin/gh/ydwu4/334/head 2025-12-04T09:21:04.4961347Z * [new branch] gh/ydwu4/334/orig -> origin/gh/ydwu4/334/orig 2025-12-04T09:21:04.4963266Z * [new branch] gh/ydwu4/335/base -> origin/gh/ydwu4/335/base 2025-12-04T09:21:04.4967592Z * [new branch] gh/ydwu4/335/head -> origin/gh/ydwu4/335/head 2025-12-04T09:21:04.4967893Z * [new branch] gh/ydwu4/335/orig -> origin/gh/ydwu4/335/orig 2025-12-04T09:21:04.4968153Z * [new branch] gh/ydwu4/337/base -> origin/gh/ydwu4/337/base 2025-12-04T09:21:04.4968899Z * [new branch] gh/ydwu4/337/head -> origin/gh/ydwu4/337/head 2025-12-04T09:21:04.4970198Z * [new branch] gh/ydwu4/337/orig -> origin/gh/ydwu4/337/orig 2025-12-04T09:21:04.4971945Z * [new branch] gh/ydwu4/339/base -> origin/gh/ydwu4/339/base 2025-12-04T09:21:04.4973180Z * [new branch] gh/ydwu4/339/head -> origin/gh/ydwu4/339/head 2025-12-04T09:21:04.4974459Z * [new branch] gh/ydwu4/339/orig -> origin/gh/ydwu4/339/orig 2025-12-04T09:21:04.4976447Z * [new branch] gh/yf225/133/base -> origin/gh/yf225/133/base 2025-12-04T09:21:04.4977624Z * [new branch] gh/yf225/133/head -> origin/gh/yf225/133/head 2025-12-04T09:21:04.4979358Z * [new branch] gh/yf225/93/base -> origin/gh/yf225/93/base 2025-12-04T09:21:04.4980559Z * [new branch] gh/yf225/93/head -> origin/gh/yf225/93/head 2025-12-04T09:21:04.4983057Z * [new branch] gh/yifuwang/152/base -> origin/gh/yifuwang/152/base 2025-12-04T09:21:04.4984678Z * [new branch] gh/yifuwang/152/head -> origin/gh/yifuwang/152/head 2025-12-04T09:21:04.4985961Z * [new branch] gh/yifuwang/152/orig -> origin/gh/yifuwang/152/orig 2025-12-04T09:21:04.4988042Z * [new branch] gh/yifuwang/195/base -> origin/gh/yifuwang/195/base 2025-12-04T09:21:04.4989208Z * [new branch] gh/yifuwang/195/head -> origin/gh/yifuwang/195/head 2025-12-04T09:21:04.4990533Z * [new branch] gh/yifuwang/195/orig -> origin/gh/yifuwang/195/orig 2025-12-04T09:21:04.4992574Z * [new branch] gh/yiming0416/1/base -> origin/gh/yiming0416/1/base 2025-12-04T09:21:04.4993787Z * [new branch] gh/yiming0416/1/head -> origin/gh/yiming0416/1/head 2025-12-04T09:21:04.4995275Z * [new branch] gh/yiming0416/2/base -> origin/gh/yiming0416/2/base 2025-12-04T09:21:04.4996453Z * [new branch] gh/yiming0416/2/head -> origin/gh/yiming0416/2/head 2025-12-04T09:21:04.4998506Z * [new branch] gh/yushangdi/1/base -> origin/gh/yushangdi/1/base 2025-12-04T09:21:04.4999875Z * [new branch] gh/yushangdi/1/head -> origin/gh/yushangdi/1/head 2025-12-04T09:21:04.5002176Z * [new branch] gh/yushangdi/10/base -> origin/gh/yushangdi/10/base 2025-12-04T09:21:04.5003492Z * [new branch] gh/yushangdi/10/head -> origin/gh/yushangdi/10/head 2025-12-04T09:21:04.5004689Z * [new branch] gh/yushangdi/10/orig -> origin/gh/yushangdi/10/orig 2025-12-04T09:21:04.5006418Z * [new branch] gh/yushangdi/11/base -> origin/gh/yushangdi/11/base 2025-12-04T09:21:04.5007632Z * [new branch] gh/yushangdi/11/head -> origin/gh/yushangdi/11/head 2025-12-04T09:21:04.5008909Z * [new branch] gh/yushangdi/11/orig -> origin/gh/yushangdi/11/orig 2025-12-04T09:21:04.5010368Z * [new branch] gh/yushangdi/2/base -> origin/gh/yushangdi/2/base 2025-12-04T09:21:04.5011574Z * [new branch] gh/yushangdi/2/head -> origin/gh/yushangdi/2/head 2025-12-04T09:21:04.5013246Z * [new branch] gh/yushangdi/7/base -> origin/gh/yushangdi/7/base 2025-12-04T09:21:04.5014527Z * [new branch] gh/yushangdi/7/head -> origin/gh/yushangdi/7/head 2025-12-04T09:21:04.5015689Z * [new branch] gh/yushangdi/7/orig -> origin/gh/yushangdi/7/orig 2025-12-04T09:21:04.5017594Z * [new branch] gh/yushangdi/8/base -> origin/gh/yushangdi/8/base 2025-12-04T09:21:04.5019025Z * [new branch] gh/yushangdi/8/head -> origin/gh/yushangdi/8/head 2025-12-04T09:21:04.5020277Z * [new branch] gh/yushangdi/8/orig -> origin/gh/yushangdi/8/orig 2025-12-04T09:21:04.5021841Z * [new branch] gh/yushangdi/9/base -> origin/gh/yushangdi/9/base 2025-12-04T09:21:04.5023298Z * [new branch] gh/yushangdi/9/head -> origin/gh/yushangdi/9/head 2025-12-04T09:21:04.5030314Z * [new branch] gh/yushangdi/9/orig -> origin/gh/yushangdi/9/orig 2025-12-04T09:21:04.5030646Z * [new branch] gh/zklaus/19/base -> origin/gh/zklaus/19/base 2025-12-04T09:21:04.5030943Z * [new branch] gh/zklaus/19/head -> origin/gh/zklaus/19/head 2025-12-04T09:21:04.5031202Z * [new branch] gh/zklaus/19/orig -> origin/gh/zklaus/19/orig 2025-12-04T09:21:04.5031500Z * [new branch] gh/zklaus/20/base -> origin/gh/zklaus/20/base 2025-12-04T09:21:04.5032565Z * [new branch] gh/zklaus/20/head -> origin/gh/zklaus/20/head 2025-12-04T09:21:04.5033770Z * [new branch] gh/zklaus/20/orig -> origin/gh/zklaus/20/orig 2025-12-04T09:21:04.5035410Z * [new branch] gh/zklaus/21/base -> origin/gh/zklaus/21/base 2025-12-04T09:21:04.5036651Z * [new branch] gh/zklaus/21/head -> origin/gh/zklaus/21/head 2025-12-04T09:21:04.5037910Z * [new branch] gh/zklaus/21/orig -> origin/gh/zklaus/21/orig 2025-12-04T09:21:04.5039710Z * [new branch] gh/zklaus/22/base -> origin/gh/zklaus/22/base 2025-12-04T09:21:04.5040975Z * [new branch] gh/zklaus/22/head -> origin/gh/zklaus/22/head 2025-12-04T09:21:04.5042594Z * [new branch] gh/zklaus/22/orig -> origin/gh/zklaus/22/orig 2025-12-04T09:21:04.5044231Z * [new branch] gh/zklaus/23/base -> origin/gh/zklaus/23/base 2025-12-04T09:21:04.5045384Z * [new branch] gh/zklaus/23/head -> origin/gh/zklaus/23/head 2025-12-04T09:21:04.5046594Z * [new branch] gh/zklaus/23/orig -> origin/gh/zklaus/23/orig 2025-12-04T09:21:04.5048076Z * [new branch] gh/zklaus/24/base -> origin/gh/zklaus/24/base 2025-12-04T09:21:04.5049241Z * [new branch] gh/zklaus/24/head -> origin/gh/zklaus/24/head 2025-12-04T09:21:04.5050412Z * [new branch] gh/zklaus/24/orig -> origin/gh/zklaus/24/orig 2025-12-04T09:21:04.5052559Z * [new branch] gh/zou3519/1197/base -> origin/gh/zou3519/1197/base 2025-12-04T09:21:04.5053647Z * [new branch] gh/zou3519/1197/head -> origin/gh/zou3519/1197/head 2025-12-04T09:21:04.5054934Z * [new branch] gh/zou3519/1197/orig -> origin/gh/zou3519/1197/orig 2025-12-04T09:21:04.5056939Z * [new branch] gh/zou3519/1199/base -> origin/gh/zou3519/1199/base 2025-12-04T09:21:04.5058141Z * [new branch] gh/zou3519/1199/head -> origin/gh/zou3519/1199/head 2025-12-04T09:21:04.5059389Z * [new branch] gh/zou3519/1199/orig -> origin/gh/zou3519/1199/orig 2025-12-04T09:21:04.5060995Z * [new branch] gh/zou3519/1200/base -> origin/gh/zou3519/1200/base 2025-12-04T09:21:04.5062191Z * [new branch] gh/zou3519/1200/head -> origin/gh/zou3519/1200/head 2025-12-04T09:21:04.5063369Z * [new branch] gh/zou3519/1200/orig -> origin/gh/zou3519/1200/orig 2025-12-04T09:21:04.5065445Z * [new branch] gh/zou3519/1201/base -> origin/gh/zou3519/1201/base 2025-12-04T09:21:04.5066705Z * [new branch] gh/zou3519/1201/head -> origin/gh/zou3519/1201/head 2025-12-04T09:21:04.5068181Z * [new branch] gh/zou3519/1201/orig -> origin/gh/zou3519/1201/orig 2025-12-04T09:21:04.5069603Z * [new branch] gh/zou3519/1202/base -> origin/gh/zou3519/1202/base 2025-12-04T09:21:04.5070898Z * [new branch] gh/zou3519/1202/head -> origin/gh/zou3519/1202/head 2025-12-04T09:21:04.5072241Z * [new branch] gh/zou3519/1202/orig -> origin/gh/zou3519/1202/orig 2025-12-04T09:21:04.5074216Z * [new branch] gh/zpcore/1/base -> origin/gh/zpcore/1/base 2025-12-04T09:21:04.5075428Z * [new branch] gh/zpcore/1/head -> origin/gh/zpcore/1/head 2025-12-04T09:21:04.5077128Z * [new branch] gh/zpcore/11/base -> origin/gh/zpcore/11/base 2025-12-04T09:21:04.5078480Z * [new branch] gh/zpcore/11/head -> origin/gh/zpcore/11/head 2025-12-04T09:21:04.5079788Z * [new branch] gh/zpcore/11/orig -> origin/gh/zpcore/11/orig 2025-12-04T09:21:04.5081830Z * [new branch] gh/zpcore/12/base -> origin/gh/zpcore/12/base 2025-12-04T09:21:04.5083488Z * [new branch] gh/zpcore/12/head -> origin/gh/zpcore/12/head 2025-12-04T09:21:04.5084822Z * [new branch] gh/zpcore/12/orig -> origin/gh/zpcore/12/orig 2025-12-04T09:21:04.5086535Z * [new branch] gh/zpcore/13/base -> origin/gh/zpcore/13/base 2025-12-04T09:21:04.5093156Z * [new branch] gh/zpcore/13/head -> origin/gh/zpcore/13/head 2025-12-04T09:21:04.5094359Z * [new branch] gh/zpcore/13/orig -> origin/gh/zpcore/13/orig 2025-12-04T09:21:04.5096508Z * [new branch] gh/zpcore/14/base -> origin/gh/zpcore/14/base 2025-12-04T09:21:04.5098166Z * [new branch] gh/zpcore/14/head -> origin/gh/zpcore/14/head 2025-12-04T09:21:04.5099493Z * [new branch] gh/zpcore/14/orig -> origin/gh/zpcore/14/orig 2025-12-04T09:21:04.5102173Z * [new branch] gh/zpcore/15/base -> origin/gh/zpcore/15/base 2025-12-04T09:21:04.5103397Z * [new branch] gh/zpcore/15/head -> origin/gh/zpcore/15/head 2025-12-04T09:21:04.5104674Z * [new branch] gh/zpcore/15/orig -> origin/gh/zpcore/15/orig 2025-12-04T09:21:04.5106406Z * [new branch] gh/zpcore/2/base -> origin/gh/zpcore/2/base 2025-12-04T09:21:04.5107957Z * [new branch] gh/zpcore/2/head -> origin/gh/zpcore/2/head 2025-12-04T09:21:04.5110217Z * [new branch] gh/zpcore/21/base -> origin/gh/zpcore/21/base 2025-12-04T09:21:04.5111629Z * [new branch] gh/zpcore/21/head -> origin/gh/zpcore/21/head 2025-12-04T09:21:04.5112816Z * [new branch] gh/zpcore/21/orig -> origin/gh/zpcore/21/orig 2025-12-04T09:21:04.5114816Z * [new branch] gh/zpcore/22/base -> origin/gh/zpcore/22/base 2025-12-04T09:21:04.5116025Z * [new branch] gh/zpcore/22/head -> origin/gh/zpcore/22/head 2025-12-04T09:21:04.5117287Z * [new branch] gh/zpcore/22/orig -> origin/gh/zpcore/22/orig 2025-12-04T09:21:04.5119074Z * [new branch] gh/zpcore/23/base -> origin/gh/zpcore/23/base 2025-12-04T09:21:04.5120390Z * [new branch] gh/zpcore/23/head -> origin/gh/zpcore/23/head 2025-12-04T09:21:04.5121562Z * [new branch] gh/zpcore/23/orig -> origin/gh/zpcore/23/orig 2025-12-04T09:21:04.5122980Z * [new branch] gh/zpcore/24/base -> origin/gh/zpcore/24/base 2025-12-04T09:21:04.5124234Z * [new branch] gh/zpcore/24/head -> origin/gh/zpcore/24/head 2025-12-04T09:21:04.5125428Z * [new branch] gh/zpcore/24/orig -> origin/gh/zpcore/24/orig 2025-12-04T09:21:04.5127192Z * [new branch] gh/zpcore/25/base -> origin/gh/zpcore/25/base 2025-12-04T09:21:04.5128336Z * [new branch] gh/zpcore/25/head -> origin/gh/zpcore/25/head 2025-12-04T09:21:04.5129501Z * [new branch] gh/zpcore/25/orig -> origin/gh/zpcore/25/orig 2025-12-04T09:21:04.5131281Z * [new branch] gh/zpcore/26/base -> origin/gh/zpcore/26/base 2025-12-04T09:21:04.5132529Z * [new branch] gh/zpcore/26/head -> origin/gh/zpcore/26/head 2025-12-04T09:21:04.5133753Z * [new branch] gh/zpcore/26/orig -> origin/gh/zpcore/26/orig 2025-12-04T09:21:04.5135582Z * [new branch] gh/zpcore/27/base -> origin/gh/zpcore/27/base 2025-12-04T09:21:04.5136770Z * [new branch] gh/zpcore/27/head -> origin/gh/zpcore/27/head 2025-12-04T09:21:04.5138189Z * [new branch] gh/zpcore/27/orig -> origin/gh/zpcore/27/orig 2025-12-04T09:21:04.5140270Z * [new branch] gh/zpcore/28/base -> origin/gh/zpcore/28/base 2025-12-04T09:21:04.5141996Z * [new branch] gh/zpcore/28/head -> origin/gh/zpcore/28/head 2025-12-04T09:21:04.5143218Z * [new branch] gh/zpcore/28/orig -> origin/gh/zpcore/28/orig 2025-12-04T09:21:04.5144740Z * [new branch] gh/zpcore/3/base -> origin/gh/zpcore/3/base 2025-12-04T09:21:04.5145845Z * [new branch] gh/zpcore/3/head -> origin/gh/zpcore/3/head 2025-12-04T09:21:04.5147798Z * [new branch] gh/zpcore/4/base -> origin/gh/zpcore/4/base 2025-12-04T09:21:04.5148953Z * [new branch] gh/zpcore/4/head -> origin/gh/zpcore/4/head 2025-12-04T09:21:04.5151276Z * [new branch] gh/zpcore/5/base -> origin/gh/zpcore/5/base 2025-12-04T09:21:04.5152420Z * [new branch] gh/zpcore/5/head -> origin/gh/zpcore/5/head 2025-12-04T09:21:04.5153897Z * [new branch] gh/zpcore/6/base -> origin/gh/zpcore/6/base 2025-12-04T09:21:04.5155184Z * [new branch] gh/zpcore/6/head -> origin/gh/zpcore/6/head 2025-12-04T09:21:04.5157146Z * [new branch] gh/zpcore/7/base -> origin/gh/zpcore/7/base 2025-12-04T09:21:04.5158268Z * [new branch] gh/zpcore/7/head -> origin/gh/zpcore/7/head 2025-12-04T09:21:04.5159814Z * [new branch] gh/zpcore/8/base -> origin/gh/zpcore/8/base 2025-12-04T09:21:04.5161060Z * [new branch] gh/zpcore/8/head -> origin/gh/zpcore/8/head 2025-12-04T09:21:04.5162546Z * [new branch] google-main -> origin/google-main 2025-12-04T09:21:04.5164354Z * [new branch] guangyey/external_stream -> origin/guangyey/external_stream 2025-12-04T09:21:04.5165559Z * [new branch] guangyey/test_2025 -> origin/guangyey/test_2025 2025-12-04T09:21:04.5167410Z * [new branch] guilhermeleobas/cherry-pick-55d87d9dfd9 -> origin/guilhermeleobas/cherry-pick-55d87d9dfd9 2025-12-04T09:21:04.5168939Z * [new branch] hameerabbasi/complex_tensor_subclass -> origin/hameerabbasi/complex_tensor_subclass 2025-12-04T09:21:04.5170278Z * [new branch] hameerabbasi/fix-ctensor-gradcheck-tests -> origin/hameerabbasi/fix-ctensor-gradcheck-tests 2025-12-04T09:21:04.5171248Z * [new branch] hameerabbasi/gradcheck-allclose -> origin/hameerabbasi/gradcheck-allclose 2025-12-04T09:21:04.5172448Z * [new branch] hc_baseline -> origin/hc_baseline 2025-12-04T09:21:04.5173708Z * [new branch] hhh_rand -> origin/hhh_rand 2025-12-04T09:21:04.5175272Z * [new branch] huba/f1 -> origin/huba/f1 2025-12-04T09:21:04.5177275Z * [new branch] increase-timeout-linux-jammy-cuda12_8-py3_10-gcc11-test -> origin/increase-timeout-linux-jammy-cuda12_8-py3_10-gcc11-test 2025-12-04T09:21:04.5178055Z * [new branch] inlining -> origin/inlining 2025-12-04T09:21:04.5179361Z * [new branch] inlining-ezyang -> origin/inlining-ezyang 2025-12-04T09:21:04.5180618Z * [new branch] install-torchao-0.13.0 -> origin/install-torchao-0.13.0 2025-12-04T09:21:04.5182351Z * [new branch] instrument-trunk-pull-linux-with-job-test-filters -> origin/instrument-trunk-pull-linux-with-job-test-filters 2025-12-04T09:21:04.5183167Z * [new branch] invoke-subgraph -> origin/invoke-subgraph 2025-12-04T09:21:04.5184539Z * [new branch] issue#58739 -> origin/issue#58739 2025-12-04T09:21:04.5185863Z * [new branch] jainapurva-patch-1 -> origin/jainapurva-patch-1 2025-12-04T09:21:04.5187770Z * [new branch] jathu/o3 -> origin/jathu/o3 2025-12-04T09:21:04.5188949Z * [new branch] jathu/sve -> origin/jathu/sve 2025-12-04T09:21:04.5191442Z * [new branch] jcaip/test-cusparselt-version-0.6.2 -> origin/jcaip/test-cusparselt-version-0.6.2 2025-12-04T09:21:04.5192244Z * [new branch] jcaip/update-cusparselt-0.6.2 -> origin/jcaip/update-cusparselt-0.6.2 2025-12-04T09:21:04.5193821Z * [new branch] jiannanWang/memorysnapshot_filter -> origin/jiannanWang/memorysnapshot_filter 2025-12-04T09:21:04.5195024Z * [new branch] jiannanWang/profilerstepwarning -> origin/jiannanWang/profilerstepwarning 2025-12-04T09:21:04.5196278Z * [new branch] jithunnair-amd-patch-1 -> origin/jithunnair-amd-patch-1 2025-12-04T09:21:04.5197613Z * [new branch] jithunnair-amd-patch-10 -> origin/jithunnair-amd-patch-10 2025-12-04T09:21:04.5199025Z * [new branch] jithunnair-amd-patch-2 -> origin/jithunnair-amd-patch-2 2025-12-04T09:21:04.5200460Z * [new branch] jithunnair-amd-patch-3 -> origin/jithunnair-amd-patch-3 2025-12-04T09:21:04.5202641Z * [new branch] jithunnair-amd-patch-4 -> origin/jithunnair-amd-patch-4 2025-12-04T09:21:04.5203903Z * [new branch] jithunnair-amd-patch-5 -> origin/jithunnair-amd-patch-5 2025-12-04T09:21:04.5205276Z * [new branch] jithunnair-amd-patch-6 -> origin/jithunnair-amd-patch-6 2025-12-04T09:21:04.5206526Z * [new branch] jithunnair-amd-patch-7 -> origin/jithunnair-amd-patch-7 2025-12-04T09:21:04.5207894Z * [new branch] jithunnair-amd-patch-8 -> origin/jithunnair-amd-patch-8 2025-12-04T09:21:04.5209154Z * [new branch] jithunnair-amd-patch-9 -> origin/jithunnair-amd-patch-9 2025-12-04T09:21:04.5210841Z * [new branch] justinchu/native-qdq -> origin/justinchu/native-qdq 2025-12-04T09:21:04.5217442Z * [new branch] kainan666/xlf_debug -> origin/kainan666/xlf_debug 2025-12-04T09:21:04.5217676Z * [new branch] kainan_test -> origin/kainan_test 2025-12-04T09:21:04.5217967Z * [new branch] larryliu0820-patch-1 -> origin/larryliu0820-patch-1 2025-12-04T09:21:04.5218323Z * [new branch] leslie/test_group_gemm_epilogues -> origin/leslie/test_group_gemm_epilogues 2025-12-04T09:21:04.5218801Z * [new branch] lessw2020/fix_cutlass_cache_error -> origin/lessw2020/fix_cutlass_cache_error 2025-12-04T09:21:04.5220361Z * [new branch] liaoxuan/shm_all_reduce -> origin/liaoxuan/shm_all_reduce 2025-12-04T09:21:04.5221613Z * [new branch] liaoxuan/test_fa_disable_softmax -> origin/liaoxuan/test_fa_disable_softmax 2025-12-04T09:21:04.5222817Z * [new branch] liaoxuan/test_int8_sdpa -> origin/liaoxuan/test_int8_sdpa 2025-12-04T09:21:04.5224040Z * [new branch] llama4-stable -> origin/llama4-stable 2025-12-04T09:21:04.5226084Z * [new branch] lts/release/1.8 -> origin/lts/release/1.8 2025-12-04T09:21:04.5228034Z * [new branch] lucaskabela/#94773 -> origin/lucaskabela/#94773 2025-12-04T09:21:04.5229217Z * [new branch] lucaskabela/fix_164876 -> origin/lucaskabela/fix_164876 2025-12-04T09:21:04.5230378Z * [new branch] lucaskabela/flop_counter -> origin/lucaskabela/flop_counter 2025-12-04T09:21:04.5231594Z * [new branch] lucaskabela/func_under_decomp -> origin/lucaskabela/func_under_decomp 2025-12-04T09:21:04.5232831Z * [new branch] lucaskabela/functional_in_dynamo -> origin/lucaskabela/functional_in_dynamo 2025-12-04T09:21:04.5234095Z * [new branch] lucaskabela/install_params_as_graph_attr -> origin/lucaskabela/install_params_as_graph_attr 2025-12-04T09:21:04.5235667Z * [new branch] lucaskabela/parameters_as_graph_attr -> origin/lucaskabela/parameters_as_graph_attr 2025-12-04T09:21:04.5237292Z * [new branch] lucaskabela/remove_aot_dispatcher_metadata -> origin/lucaskabela/remove_aot_dispatcher_metadata 2025-12-04T09:21:04.5238472Z * [new branch] lucaskabela/rnn_decomp -> origin/lucaskabela/rnn_decomp 2025-12-04T09:21:04.5239731Z * [new branch] lucaskabela/typing_backends -> origin/lucaskabela/typing_backends 2025-12-04T09:21:04.5240997Z * [new branch] lucaskabela/typing_ctx_manager -> origin/lucaskabela/typing_ctx_manager 2025-12-04T09:21:04.5242194Z * [new branch] lucaskabela/typing_nn_module -> origin/lucaskabela/typing_nn_module 2025-12-04T09:21:04.5243679Z * [new branch] lucaskabela/typing_user_defined -> origin/lucaskabela/typing_user_defined 2025-12-04T09:21:04.5244861Z * [new branch] lucaskabela/typing_variables -> origin/lucaskabela/typing_variables 2025-12-04T09:21:04.5246114Z * [new branch] lucaskabela/typing_variables_dicts -> origin/lucaskabela/typing_variables_dicts 2025-12-04T09:21:04.5247325Z * [new branch] lucaskabela/typing_variables_functions -> origin/lucaskabela/typing_variables_functions 2025-12-04T09:21:04.5248913Z * [new branch] lucaskabela/typing_variables_lists -> origin/lucaskabela/typing_variables_lists 2025-12-04T09:21:04.5250473Z * [new branch] lw/torch_box_by_ref -> origin/lw/torch_box_by_ref 2025-12-04T09:21:04.5251755Z * [new branch] main -> origin/main 2025-12-04T09:21:04.5253157Z * [new branch] malfet-patch-1 -> origin/malfet-patch-1 2025-12-04T09:21:04.5254539Z * [new branch] malfet-patch-2 -> origin/malfet-patch-2 2025-12-04T09:21:04.5255948Z * [new branch] malfet-patch-3 -> origin/malfet-patch-3 2025-12-04T09:21:04.5257306Z * [new branch] malfet-patch-4 -> origin/malfet-patch-4 2025-12-04T09:21:04.5258609Z * [new branch] malfet-patch-5 -> origin/malfet-patch-5 2025-12-04T09:21:04.5259835Z * [new branch] malfet-patch-6 -> origin/malfet-patch-6 2025-12-04T09:21:04.5261200Z * [new branch] malfet-patch-7 -> origin/malfet-patch-7 2025-12-04T09:21:04.5262480Z * [new branch] malfet-patch-8 -> origin/malfet-patch-8 2025-12-04T09:21:04.5264109Z * [new branch] malfet/add-3.14-ci -> origin/malfet/add-3.14-ci 2025-12-04T09:21:04.5265565Z * [new branch] malfet/be-do-not-make-typos-in-build-artifacts -> origin/malfet/be-do-not-make-typos-in-build-artifacts 2025-12-04T09:21:04.5266836Z * [new branch] malfet/be-move-more-settings-to-checkout-pytorch -> origin/malfet/be-move-more-settings-to-checkout-pytorch 2025-12-04T09:21:04.5268477Z * [new branch] malfet/be-remove-misisng-neon-headers -> origin/malfet/be-remove-misisng-neon-headers 2025-12-04T09:21:04.5269778Z * [new branch] malfet/mps-implement-col2im -> origin/malfet/mps-implement-col2im 2025-12-04T09:21:04.5271605Z * [new branch] manuel/aoti_metal_shimify-thread_safe -> origin/manuel/aoti_metal_shimify-thread_safe 2025-12-04T09:21:04.5272702Z * [new branch] manuel/inductor_link_openmp -> origin/manuel/inductor_link_openmp 2025-12-04T09:21:04.5280060Z * [new branch] masnesral/metaconda -> origin/masnesral/metaconda 2025-12-04T09:21:04.5285770Z * [new branch] mem_profiler_flaky_fix -> origin/mem_profiler_flaky_fix 2025-12-04T09:21:04.5287047Z * [new branch] mem_profiler_stack_trace -> origin/mem_profiler_stack_trace 2025-12-04T09:21:04.5288262Z * [new branch] memory_profiler_stack -> origin/memory_profiler_stack 2025-12-04T09:21:04.5289540Z * [new branch] metascroy-patch-1 -> origin/metascroy-patch-1 2025-12-04T09:21:04.5290788Z * [new branch] mingw_posix -> origin/mingw_posix 2025-12-04T09:21:04.5292492Z * [new branch] mlazos/S429861-debug -> origin/mlazos/S429861-debug 2025-12-04T09:21:04.5293645Z * [new branch] mlazos/aa -> origin/mlazos/aa 2025-12-04T09:21:04.5294786Z * [new branch] mlazos/acts -> origin/mlazos/acts 2025-12-04T09:21:04.5296004Z * [new branch] mlazos/arg-renames -> origin/mlazos/arg-renames 2025-12-04T09:21:04.5297146Z * [new branch] mlazos/bad-cudagraphs -> origin/mlazos/bad-cudagraphs 2025-12-04T09:21:04.5298342Z * [new branch] mlazos/baseline-graph-breaks -> origin/mlazos/baseline-graph-breaks 2025-12-04T09:21:04.5299381Z * [new branch] mlazos/beta-tensor -> origin/mlazos/beta-tensor 2025-12-04T09:21:04.5300480Z * [new branch] mlazos/buffers -> origin/mlazos/buffers 2025-12-04T09:21:04.5310967Z * [new branch] mlazos/buffers2 -> origin/mlazos/buffers2 2025-12-04T09:21:04.5312656Z * [new branch] mlazos/buffers3 -> origin/mlazos/buffers3 2025-12-04T09:21:04.5314144Z * [new branch] mlazos/bwd -> origin/mlazos/bwd 2025-12-04T09:21:04.5315321Z * [new branch] mlazos/combo-test -> origin/mlazos/combo-test 2025-12-04T09:21:04.5316585Z * [new branch] mlazos/ctx-cleanup -> origin/mlazos/ctx-cleanup 2025-12-04T09:21:04.5317832Z * [new branch] mlazos/cuda-cmd-log -> origin/mlazos/cuda-cmd-log 2025-12-04T09:21:04.5319341Z * [new branch] mlazos/cudagraph-tests -> origin/mlazos/cudagraph-tests 2025-12-04T09:21:04.5320593Z * [new branch] mlazos/cudagraphs-measurement -> origin/mlazos/cudagraphs-measurement 2025-12-04T09:21:04.5321774Z * [new branch] mlazos/cutlass-test -> origin/mlazos/cutlass-test 2025-12-04T09:21:04.5323074Z * [new branch] mlazos/cutlass-topo-bug -> origin/mlazos/cutlass-topo-bug 2025-12-04T09:21:04.5324193Z * [new branch] mlazos/dataclass-proxy -> origin/mlazos/dataclass-proxy 2025-12-04T09:21:04.5325347Z * [new branch] mlazos/dc-attrs -> origin/mlazos/dc-attrs 2025-12-04T09:21:04.5326537Z * [new branch] mlazos/dc-helion -> origin/mlazos/dc-helion 2025-12-04T09:21:04.5327717Z * [new branch] mlazos/dict-fix -> origin/mlazos/dict-fix 2025-12-04T09:21:04.5328941Z * [new branch] mlazos/disable-tf -> origin/mlazos/disable-tf 2025-12-04T09:21:04.5330578Z * [new branch] mlazos/dupe-fix -> origin/mlazos/dupe-fix 2025-12-04T09:21:04.5331840Z * [new branch] mlazos/dyn-batch -> origin/mlazos/dyn-batch 2025-12-04T09:21:04.5333103Z * [new branch] mlazos/evt -> origin/mlazos/evt 2025-12-04T09:21:04.5334365Z * [new branch] mlazos/extract-examples -> origin/mlazos/extract-examples 2025-12-04T09:21:04.5335527Z * [new branch] mlazos/foreach-op -> origin/mlazos/foreach-op 2025-12-04T09:21:04.5338629Z * [new branch] mlazos/fp8 -> origin/mlazos/fp8 2025-12-04T09:21:04.5341926Z * [new branch] mlazos/fp8-bias -> origin/mlazos/fp8-bias 2025-12-04T09:21:04.5342208Z * [new branch] mlazos/fp8-bias-fusion -> origin/mlazos/fp8-bias-fusion 2025-12-04T09:21:04.5342464Z * [new branch] mlazos/fp8-fixes -> origin/mlazos/fp8-fixes 2025-12-04T09:21:04.5342716Z * [new branch] mlazos/freezing -> origin/mlazos/freezing 2025-12-04T09:21:04.5343682Z * [new branch] mlazos/h-comp -> origin/mlazos/h-comp 2025-12-04T09:21:04.5345074Z * [new branch] mlazos/h-comp2 -> origin/mlazos/h-comp2 2025-12-04T09:21:04.5346273Z * [new branch] mlazos/hash-hop -> origin/mlazos/hash-hop 2025-12-04T09:21:04.5347705Z * [new branch] mlazos/hc -> origin/mlazos/hc 2025-12-04T09:21:04.5348978Z * [new branch] mlazos/hc-cycles -> origin/mlazos/hc-cycles 2025-12-04T09:21:04.5350201Z * [new branch] mlazos/hc-fixes -> origin/mlazos/hc-fixes 2025-12-04T09:21:04.5351384Z * [new branch] mlazos/hc-fixes3 -> origin/mlazos/hc-fixes3 2025-12-04T09:21:04.5352585Z * [new branch] mlazos/hc-fixes4 -> origin/mlazos/hc-fixes4 2025-12-04T09:21:04.5353957Z * [new branch] mlazos/hc-hf -> origin/mlazos/hc-hf 2025-12-04T09:21:04.5355161Z * [new branch] mlazos/hc-mut -> origin/mlazos/hc-mut 2025-12-04T09:21:04.5356384Z * [new branch] mlazos/hc10 -> origin/mlazos/hc10 2025-12-04T09:21:04.5357575Z * [new branch] mlazos/hc11 -> origin/mlazos/hc11 2025-12-04T09:21:04.5358855Z * [new branch] mlazos/hc12 -> origin/mlazos/hc12 2025-12-04T09:21:04.5360278Z * [new branch] mlazos/hc13 -> origin/mlazos/hc13 2025-12-04T09:21:04.5361873Z * [new branch] mlazos/hc14 -> origin/mlazos/hc14 2025-12-04T09:21:04.5363088Z * [new branch] mlazos/hc15 -> origin/mlazos/hc15 2025-12-04T09:21:04.5364258Z * [new branch] mlazos/hc2 -> origin/mlazos/hc2 2025-12-04T09:21:04.5365455Z * [new branch] mlazos/hc4 -> origin/mlazos/hc4 2025-12-04T09:21:04.5366586Z * [new branch] mlazos/hc5 -> origin/mlazos/hc5 2025-12-04T09:21:04.5367741Z * [new branch] mlazos/hc6 -> origin/mlazos/hc6 2025-12-04T09:21:04.5368999Z * [new branch] mlazos/hc7 -> origin/mlazos/hc7 2025-12-04T09:21:04.5370116Z * [new branch] mlazos/hc8 -> origin/mlazos/hc8 2025-12-04T09:21:04.5371274Z * [new branch] mlazos/hc9 -> origin/mlazos/hc9 2025-12-04T09:21:04.5372473Z * [new branch] mlazos/hc_baseline2 -> origin/mlazos/hc_baseline2 2025-12-04T09:21:04.5373703Z * [new branch] mlazos/inductor-streams -> origin/mlazos/inductor-streams 2025-12-04T09:21:04.5374681Z * [new branch] mlazos/main -> origin/mlazos/main 2025-12-04T09:21:04.5375865Z * [new branch] mlazos/mcg2 -> origin/mlazos/mcg2 2025-12-04T09:21:04.5377167Z * [new branch] mlazos/meta-guards -> origin/mlazos/meta-guards 2025-12-04T09:21:04.5378920Z * [new branch] mlazos/mlazos/foreach-map-adam -> origin/mlazos/mlazos/foreach-map-adam 2025-12-04T09:21:04.5380200Z * [new branch] mlazos/mlazos/tf-mode-backup -> origin/mlazos/mlazos/tf-mode-backup 2025-12-04T09:21:04.5381324Z * [new branch] mlazos/mod-fix -> origin/mlazos/mod-fix 2025-12-04T09:21:04.5382560Z * [new branch] mlazos/mode-fix -> origin/mlazos/mode-fix 2025-12-04T09:21:04.5383712Z * [new branch] mlazos/offsets -> origin/mlazos/offsets 2025-12-04T09:21:04.5384813Z * [new branch] mlazos/overguarding -> origin/mlazos/overguarding 2025-12-04T09:21:04.5386019Z * [new branch] mlazos/proxy-ctors -> origin/mlazos/proxy-ctors 2025-12-04T09:21:04.5387558Z * [new branch] mlazos/quant-fix -> origin/mlazos/quant-fix 2025-12-04T09:21:04.5388895Z * [new branch] mlazos/resnet-fix -> origin/mlazos/resnet-fix 2025-12-04T09:21:04.5390152Z * [new branch] mlazos/rm-buf-names -> origin/mlazos/rm-buf-names 2025-12-04T09:21:04.5391353Z * [new branch] mlazos/rm-code -> origin/mlazos/rm-code 2025-12-04T09:21:04.5392659Z * [new branch] mlazos/rm-spam -> origin/mlazos/rm-spam 2025-12-04T09:21:04.5393906Z * [new branch] mlazos/rtp -> origin/mlazos/rtp 2025-12-04T09:21:04.5395191Z * [new branch] mlazos/static-idx-dbg -> origin/mlazos/static-idx-dbg 2025-12-04T09:21:04.5396421Z * [new branch] mlazos/static-inputs-log -> origin/mlazos/static-inputs-log 2025-12-04T09:21:04.5397472Z * [new branch] mlazos/stests -> origin/mlazos/stests 2025-12-04T09:21:04.5398705Z * [new branch] mlazos/stream-ops -> origin/mlazos/stream-ops 2025-12-04T09:21:04.5400247Z * [new branch] mlazos/td-fix2 -> origin/mlazos/td-fix2 2025-12-04T09:21:04.5402030Z * [new branch] mlazos/tensor-hasattr2 -> origin/mlazos/tensor-hasattr2 2025-12-04T09:21:04.5403188Z * [new branch] mlazos/test -> origin/mlazos/test 2025-12-04T09:21:04.5404544Z * [new branch] mlazos/tf-mode -> origin/mlazos/tf-mode 2025-12-04T09:21:04.5406017Z * [new branch] mlazos/tf-mode-backup2 -> origin/mlazos/tf-mode-backup2 2025-12-04T09:21:04.5407261Z * [new branch] mlazos/tf-mode-reland -> origin/mlazos/tf-mode-reland 2025-12-04T09:21:04.5408641Z * [new branch] mlazos/tf-mode-reland2 -> origin/mlazos/tf-mode-reland2 2025-12-04T09:21:04.5409867Z * [new branch] mlazos/tf-mode-reland3 -> origin/mlazos/tf-mode-reland3 2025-12-04T09:21:04.5411078Z * [new branch] mlazos/triton-no-epi -> origin/mlazos/triton-no-epi 2025-12-04T09:21:04.5412295Z * [new branch] mlazos/tune-proto -> origin/mlazos/tune-proto 2025-12-04T09:21:04.5413640Z * [new branch] mlazos/tuple-fixes -> origin/mlazos/tuple-fixes 2025-12-04T09:21:04.5415458Z * [new branch] mlazos/tuple-fixes2 -> origin/mlazos/tuple-fixes2 2025-12-04T09:21:04.5416674Z * [new branch] mlazos/tuple-handling -> origin/mlazos/tuple-handling 2025-12-04T09:21:04.5417933Z * [new branch] mlazos/user-stream-base -> origin/mlazos/user-stream-base 2025-12-04T09:21:04.5419098Z * [new branch] mlazos/user-streams -> origin/mlazos/user-streams 2025-12-04T09:21:04.5420378Z * [new branch] mlazos/user-streams-backup -> origin/mlazos/user-streams-backup 2025-12-04T09:21:04.5421567Z * [new branch] mlazos/user-streams-backup2 -> origin/mlazos/user-streams-backup2 2025-12-04T09:21:04.5422714Z * [new branch] mlazos/vary-beta -> origin/mlazos/vary-beta 2025-12-04T09:21:04.5423910Z * [new branch] mlazos/vary-beta2 -> origin/mlazos/vary-beta2 2025-12-04T09:21:04.5425170Z * [new branch] mlazos/weird-perf1 -> origin/mlazos/weird-perf1 2025-12-04T09:21:04.5426435Z * [new branch] mm_out_dtype_compile -> origin/mm_out_dtype_compile 2025-12-04T09:21:04.5428050Z * [new branch] module-shim -> origin/module-shim 2025-12-04T09:21:04.5429382Z * [new branch] move_config -> origin/move_config 2025-12-04T09:21:04.5431063Z * [new branch] msaroufim/reduce -> origin/msaroufim/reduce 2025-12-04T09:21:04.5432798Z * [new branch] mtia/basic-cmake -> origin/mtia/basic-cmake 2025-12-04T09:21:04.5434546Z * [new branch] mwizak/fix-triton-block-shape -> origin/mwizak/fix-triton-block-shape 2025-12-04T09:21:04.5435738Z * [new branch] my_varlen_backup -> origin/my_varlen_backup 2025-12-04T09:21:04.5437080Z * [new branch] nativert_num_outputs -> origin/nativert_num_outputs 2025-12-04T09:21:04.5438747Z * [new branch] new-codegen -> origin/new-codegen 2025-12-04T09:21:04.5440224Z * [new branch] newtest-base -> origin/newtest-base 2025-12-04T09:21:04.5441805Z * [new branch] ngimel/addmm_dtype -> origin/ngimel/addmm_dtype 2025-12-04T09:21:04.5442918Z * [new branch] ngimel/div_inv -> origin/ngimel/div_inv 2025-12-04T09:21:04.5444069Z * [new branch] ngimel/error_index_list -> origin/ngimel/error_index_list 2025-12-04T09:21:04.5445279Z * [new branch] ngimel/gather_grid -> origin/ngimel/gather_grid 2025-12-04T09:21:04.5446498Z * [new branch] ngimel/gather_grid_release -> origin/ngimel/gather_grid_release 2025-12-04T09:21:04.5447527Z * [new branch] ngimel/gg_new -> origin/ngimel/gg_new 2025-12-04T09:21:04.5448685Z * [new branch] ngimel/hostalloc -> origin/ngimel/hostalloc 2025-12-04T09:21:04.5449801Z * [new branch] ngimel/storage_id -> origin/ngimel/storage_id 2025-12-04T09:21:04.5451164Z * [new branch] nightly -> origin/nightly 2025-12-04T09:21:04.5452961Z * [new branch] nikitaved/addmm_1_rowcol_lt_path_check -> origin/nikitaved/addmm_1_rowcol_lt_path_check 2025-12-04T09:21:04.5454160Z * [new branch] nikitaved/addmm_epilogue_fusions_2d_bias -> origin/nikitaved/addmm_epilogue_fusions_2d_bias 2025-12-04T09:21:04.5455647Z * [new branch] nikitaved/addmm_epilogue_fusions_inductor -> origin/nikitaved/addmm_epilogue_fusions_inductor 2025-12-04T09:21:04.5457073Z * [new branch] nikitaved/addmm_epilogue_fusions_scratch -> origin/nikitaved/addmm_epilogue_fusions_scratch 2025-12-04T09:21:04.5458523Z * [new branch] nikitaved/grad_addmm_epilogue_fusions -> origin/nikitaved/grad_addmm_epilogue_fusions 2025-12-04T09:21:04.5459958Z * [new branch] nikitaved/simpler_can_use_32bit_index -> origin/nikitaved/simpler_can_use_32bit_index 2025-12-04T09:21:04.5461128Z * [new branch] nikitaved/test -> origin/nikitaved/test 2025-12-04T09:21:04.5463216Z * [new branch] nmacchioni-perf-test-async-autotune -> origin/nmacchioni-perf-test-async-autotune 2025-12-04T09:21:04.5467897Z * [new branch] no_distributed_log_spew -> origin/no_distributed_log_spew 2025-12-04T09:21:04.5468147Z * [new branch] nofun-hack -> origin/nofun-hack 2025-12-04T09:21:04.5468362Z * [new branch] norm_bench -> origin/norm_bench 2025-12-04T09:21:04.5469355Z * [new branch] nullplay/fuse_matmul -> origin/nullplay/fuse_matmul 2025-12-04T09:21:04.5470696Z * [new branch] nullplay_fuse_matmul -> origin/nullplay_fuse_matmul 2025-12-04T09:21:04.5472522Z * [new branch] optimizer_test -> origin/optimizer_test 2025-12-04T09:21:04.5475137Z * [new branch] orig/release/1.10 -> origin/orig/release/1.10 2025-12-04T09:21:04.5476504Z * [new branch] orig/release/1.11 -> origin/orig/release/1.11 2025-12-04T09:21:04.5477783Z * [new branch] orig/release/1.12 -> origin/orig/release/1.12 2025-12-04T09:21:04.5479363Z * [new branch] orig/release/1.13 -> origin/orig/release/1.13 2025-12-04T09:21:04.5480650Z * [new branch] orig/release/1.6 -> origin/orig/release/1.6 2025-12-04T09:21:04.5482090Z * [new branch] orig/release/1.7 -> origin/orig/release/1.7 2025-12-04T09:21:04.5483314Z * [new branch] orig/release/1.8 -> origin/orig/release/1.8 2025-12-04T09:21:04.5484607Z * [new branch] orig/release/1.9 -> origin/orig/release/1.9 2025-12-04T09:21:04.5485803Z * [new branch] orig/release/2.0 -> origin/orig/release/2.0 2025-12-04T09:21:04.5487019Z * [new branch] orig/release/2.1 -> origin/orig/release/2.1 2025-12-04T09:21:04.5488250Z * [new branch] orig/release/2.2 -> origin/orig/release/2.2 2025-12-04T09:21:04.5489408Z * [new branch] orig/release/2.3 -> origin/orig/release/2.3 2025-12-04T09:21:04.5490583Z * [new branch] orig/release/2.4 -> origin/orig/release/2.4 2025-12-04T09:21:04.5491843Z * [new branch] orig/release/2.5 -> origin/orig/release/2.5 2025-12-04T09:21:04.5493052Z * [new branch] orig/release/2.6 -> origin/orig/release/2.6 2025-12-04T09:21:04.5494528Z * [new branch] orig/release/2.7 -> origin/orig/release/2.7 2025-12-04T09:21:04.5496158Z * [new branch] orig/release/2.8 -> origin/orig/release/2.8 2025-12-04T09:21:04.5497388Z * [new branch] orig/release/2.9 -> origin/orig/release/2.9 2025-12-04T09:21:04.5499960Z * [new branch] origin/gh/fxdawnn/1/base -> origin/origin/gh/fxdawnn/1/base 2025-12-04T09:21:04.5501675Z * [new branch] origin/gh/fxdawnn/1/orig -> origin/origin/gh/fxdawnn/1/orig 2025-12-04T09:21:04.5503924Z * [new branch] origin/gh/zpcore/14/orig -> origin/origin/gh/zpcore/14/orig 2025-12-04T09:21:04.5505348Z * [new branch] oulgen-patch-1 -> origin/oulgen-patch-1 2025-12-04T09:21:04.5506847Z * [new branch] oulgen-patch-2 -> origin/oulgen-patch-2 2025-12-04T09:21:04.5508535Z * [new branch] oulgen-patch-3 -> origin/oulgen-patch-3 2025-12-04T09:21:04.5509913Z * [new branch] oulgen-patch-4 -> origin/oulgen-patch-4 2025-12-04T09:21:04.5511232Z * [new branch] padded-tensor -> origin/padded-tensor 2025-12-04T09:21:04.5512556Z * [new branch] pca2 -> origin/pca2 2025-12-04T09:21:04.5513973Z * [new branch] per_channel_backup -> origin/per_channel_backup 2025-12-04T09:21:04.5515377Z * [new branch] perf_ops -> origin/perf_ops 2025-12-04T09:21:04.5516599Z * [new branch] perf_ops_2_9 -> origin/perf_ops_2_9 2025-12-04T09:21:04.5518066Z * [new branch] pianpwk-patch-1 -> origin/pianpwk-patch-1 2025-12-04T09:21:04.5519860Z * [new branch] pianpwk/__draft_debug_mode -> origin/pianpwk/__draft_debug_mode 2025-12-04T09:21:04.5521078Z * [new branch] pianpwk/_debug_mode_for_triton_draft -> origin/pianpwk/_debug_mode_for_triton_draft 2025-12-04T09:21:04.5522187Z * [new branch] pianpwk/_debug_nn_module_compile -> origin/pianpwk/_debug_nn_module_compile 2025-12-04T09:21:04.5523239Z * [new branch] pianpwk/_draft_triton_11_3 -> origin/pianpwk/_draft_triton_11_3 2025-12-04T09:21:04.5529795Z * [new branch] pianpwk/_manual_bucket_draft -> origin/pianpwk/_manual_bucket_draft 2025-12-04T09:21:04.5531295Z * [new branch] pianpwk/_profile_w_dispatch_keys -> origin/pianpwk/_profile_w_dispatch_keys 2025-12-04T09:21:04.5532794Z * [new branch] pianpwk/_super_draft_debug_mode -> origin/pianpwk/_super_draft_debug_mode 2025-12-04T09:21:04.5534284Z * [new branch] pianpwk/_unbacked_local_shard_size -> origin/pianpwk/_unbacked_local_shard_size 2025-12-04T09:21:04.5535420Z * [new branch] pianpwk/anomaly_tb -> origin/pianpwk/anomaly_tb 2025-12-04T09:21:04.5536820Z * [new branch] pianpwk/auto_fx_annotate -> origin/pianpwk/auto_fx_annotate 2025-12-04T09:21:04.5538071Z * [new branch] pianpwk/backed_size_oblivious_export -> origin/pianpwk/backed_size_oblivious_export 2025-12-04T09:21:04.5539155Z * [new branch] pianpwk/bert_dynamic_perf -> origin/pianpwk/bert_dynamic_perf 2025-12-04T09:21:04.5540508Z * [new branch] pianpwk/debug_fwd_stack_traces -> origin/pianpwk/debug_fwd_stack_traces 2025-12-04T09:21:04.5541781Z * [new branch] pianpwk/debug_hash_tensor -> origin/pianpwk/debug_hash_tensor 2025-12-04T09:21:04.5543036Z * [new branch] pianpwk/debug_mode_annotate -> origin/pianpwk/debug_mode_annotate 2025-12-04T09:21:04.5544198Z * [new branch] pianpwk/debug_mode_defaults -> origin/pianpwk/debug_mode_defaults 2025-12-04T09:21:04.5545403Z * [new branch] pianpwk/debug_mode_hacks -> origin/pianpwk/debug_mode_hacks 2025-12-04T09:21:04.5546784Z * [new branch] pianpwk/debug_mode_opcall_refactor -> origin/pianpwk/debug_mode_opcall_refactor 2025-12-04T09:21:04.5548199Z * [new branch] pianpwk/debug_mode_show_ids -> origin/pianpwk/debug_mode_show_ids 2025-12-04T09:21:04.5549404Z * [new branch] pianpwk/debug_mode_triton -> origin/pianpwk/debug_mode_triton 2025-12-04T09:21:04.5550732Z * [new branch] pianpwk/debug_show_stack_trace -> origin/pianpwk/debug_show_stack_trace 2025-12-04T09:21:04.5551995Z * [new branch] pianpwk/debug_wait_on_collective -> origin/pianpwk/debug_wait_on_collective 2025-12-04T09:21:04.5553243Z * [new branch] pianpwk/debugmode_compile_tf -> origin/pianpwk/debugmode_compile_tf 2025-12-04T09:21:04.5554701Z * [new branch] pianpwk/dispatch_key_debugging_for_debug -> origin/pianpwk/dispatch_key_debugging_for_debug 2025-12-04T09:21:04.5555848Z * [new branch] pianpwk/draft_debug_mode_tfcompile -> origin/pianpwk/draft_debug_mode_tfcompile 2025-12-04T09:21:04.5557026Z * [new branch] pianpwk/draft_multikernel_nn -> origin/pianpwk/draft_multikernel_nn 2025-12-04T09:21:04.5558367Z * [new branch] pianpwk/draft_multikernel_status_10_5 -> origin/pianpwk/draft_multikernel_status_10_5 2025-12-04T09:21:04.5559752Z * [new branch] pianpwk/dtensor_custom_chunk -> origin/pianpwk/dtensor_custom_chunk 2025-12-04T09:21:04.5561057Z * [new branch] pianpwk/dtensor_unbacked_keypath -> origin/pianpwk/dtensor_unbacked_keypath 2025-12-04T09:21:04.5562383Z * [new branch] pianpwk/event_list_tree -> origin/pianpwk/event_list_tree 2025-12-04T09:21:04.5563572Z * [new branch] pianpwk/false_numel_refs -> origin/pianpwk/false_numel_refs 2025-12-04T09:21:04.5565006Z * [new branch] pianpwk/maybe_guard_rel -> origin/pianpwk/maybe_guard_rel 2025-12-04T09:21:04.5566729Z * [new branch] pianpwk/multikernel_hints_draft -> origin/pianpwk/multikernel_hints_draft 2025-12-04T09:21:04.5568021Z * [new branch] pianpwk/no_size_oblivious_slice_scat -> origin/pianpwk/no_size_oblivious_slice_scat 2025-12-04T09:21:04.5569235Z * [new branch] pianpwk/oblivious_reshape_view_better -> origin/pianpwk/oblivious_reshape_view_better 2025-12-04T09:21:04.5570775Z * [new branch] pianpwk/pre_forward_hook -> origin/pianpwk/pre_forward_hook 2025-12-04T09:21:04.5572062Z * [new branch] pianpwk/skip_python_keys_alternate -> origin/pianpwk/skip_python_keys_alternate 2025-12-04T09:21:04.5573266Z * [new branch] pianpwk/skip_python_keys_in_guards -> origin/pianpwk/skip_python_keys_in_guards 2025-12-04T09:21:04.5574403Z * [new branch] pianpwk/sym_tokens_draft -> origin/pianpwk/sym_tokens_draft 2025-12-04T09:21:04.5575717Z * [new branch] pianpwk/symint_one_hot -> origin/pianpwk/symint_one_hot 2025-12-04T09:21:04.5577103Z * [new branch] pianpwk/test_pointwise_guard_or_false -> origin/pianpwk/test_pointwise_guard_or_false 2025-12-04T09:21:04.5578206Z * [new branch] pianpwk/totally_draft_sym_wrap -> origin/pianpwk/totally_draft_sym_wrap 2025-12-04T09:21:04.5579369Z * [new branch] pianpwk/try_dumb_stuff -> origin/pianpwk/try_dumb_stuff 2025-12-04T09:21:04.5580554Z * [new branch] pianpwk/try_dumb_stuff_2 -> origin/pianpwk/try_dumb_stuff_2 2025-12-04T09:21:04.5581819Z * [new branch] pianpwk/unbacked_dtensor_mm -> origin/pianpwk/unbacked_dtensor_mm 2025-12-04T09:21:04.5582964Z * [new branch] pianpwk/unbacked_tracing_12_2 -> origin/pianpwk/unbacked_tracing_12_2 2025-12-04T09:21:04.5584062Z * [new branch] pianpwk/user_symints -> origin/pianpwk/user_symints 2025-12-04T09:21:04.5585231Z * [new branch] pianpwk/wan21_reshape -> origin/pianpwk/wan21_reshape 2025-12-04T09:21:04.5592084Z * [new branch] piz/fix_partial_backward_1112 -> origin/piz/fix_partial_backward_1112 2025-12-04T09:21:04.5593852Z * [new branch] piz/prop_cache_clean -> origin/piz/prop_cache_clean 2025-12-04T09:21:04.5595182Z * [new branch] pool-separate -> origin/pool-separate 2025-12-04T09:21:04.5596508Z * [new branch] pr-156087 -> origin/pr-156087 2025-12-04T09:21:04.5598256Z * [new branch] pr/131860 -> origin/pr/131860 2025-12-04T09:21:04.5599705Z * [new branch] predispatch_to -> origin/predispatch_to 2025-12-04T09:21:04.5601387Z * [new branch] protect-c17 -> origin/protect-c17 2025-12-04T09:21:04.5603585Z * [new branch] pt-opt-cuda3 -> origin/pt-opt-cuda3 2025-12-04T09:21:04.5605460Z * [new branch] python_compiled_autograd -> origin/python_compiled_autograd 2025-12-04T09:21:04.5607430Z * [new branch] q1l1/fix_device_moved_constant_type_unknown -> origin/q1l1/fix_device_moved_constant_type_unknown 2025-12-04T09:21:04.5609241Z * [new branch] q1l1/fix_wrong_default_type_for_kernel_call_args -> origin/q1l1/fix_wrong_default_type_for_kernel_call_args 2025-12-04T09:21:04.5611269Z * [new branch] qchip/export-D54134695 -> origin/qchip/export-D54134695 2025-12-04T09:21:04.5612706Z * [new branch] quote-pytest_cache -> origin/quote-pytest_cache 2025-12-04T09:21:04.5614392Z * [new branch] reland-accgrad-stream-warn -> origin/reland-accgrad-stream-warn 2025-12-04T09:21:04.5616156Z * [new branch] release/1.10 -> origin/release/1.10 2025-12-04T09:21:04.5617367Z * [new branch] release/1.11 -> origin/release/1.11 2025-12-04T09:21:04.5618646Z * [new branch] release/1.12 -> origin/release/1.12 2025-12-04T09:21:04.5619849Z * [new branch] release/1.13 -> origin/release/1.13 2025-12-04T09:21:04.5621094Z * [new branch] release/1.4 -> origin/release/1.4 2025-12-04T09:21:04.5622548Z * [new branch] release/1.4.1 -> origin/release/1.4.1 2025-12-04T09:21:04.5623786Z * [new branch] release/1.5 -> origin/release/1.5 2025-12-04T09:21:04.5625070Z * [new branch] release/1.6 -> origin/release/1.6 2025-12-04T09:21:04.5626372Z * [new branch] release/1.7 -> origin/release/1.7 2025-12-04T09:21:04.5628105Z * [new branch] release/1.8 -> origin/release/1.8 2025-12-04T09:21:04.5629366Z * [new branch] release/1.9 -> origin/release/1.9 2025-12-04T09:21:04.5630570Z * [new branch] release/2.0 -> origin/release/2.0 2025-12-04T09:21:04.5631946Z * [new branch] release/2.1 -> origin/release/2.1 2025-12-04T09:21:04.5633269Z * [new branch] release/2.2 -> origin/release/2.2 2025-12-04T09:21:04.5634816Z * [new branch] release/2.3 -> origin/release/2.3 2025-12-04T09:21:04.5636619Z * [new branch] release/2.4 -> origin/release/2.4 2025-12-04T09:21:04.5638264Z * [new branch] release/2.5 -> origin/release/2.5 2025-12-04T09:21:04.5639735Z * [new branch] release/2.6 -> origin/release/2.6 2025-12-04T09:21:04.5641095Z * [new branch] release/2.7 -> origin/release/2.7 2025-12-04T09:21:04.5642344Z * [new branch] release/2.8 -> origin/release/2.8 2025-12-04T09:21:04.5643759Z * [new branch] release/2.9 -> origin/release/2.9 2025-12-04T09:21:04.5645048Z * [new branch] release_notes -> origin/release_notes 2025-12-04T09:21:04.5646407Z * [new branch] remove_pyinterpreter -> origin/remove_pyinterpreter 2025-12-04T09:21:04.5647989Z * [new branch] replace-pytorch-labs-20250812-195836 -> origin/replace-pytorch-labs-20250812-195836 2025-12-04T09:21:04.5649188Z * [new branch] replace-pytorch-labs-20250812-200248 -> origin/replace-pytorch-labs-20250812-200248 2025-12-04T09:21:04.5653853Z * [new branch] replace-pytorch-labs-20250812-200324 -> origin/replace-pytorch-labs-20250812-200324 2025-12-04T09:21:04.5654285Z * [new branch] replace-pytorch-labs-20250812-204020 -> origin/replace-pytorch-labs-20250812-204020 2025-12-04T09:21:04.5654877Z * [new branch] revert-131069-gh/krzysztofjordan/1/head -> origin/revert-131069-gh/krzysztofjordan/1/head 2025-12-04T09:21:04.5656900Z * [new branch] revert-131469-gh/andrewor14/51/head -> origin/revert-131469-gh/andrewor14/51/head 2025-12-04T09:21:04.5659413Z * [new branch] revert-152361-gh/fadara01/1/head -> origin/revert-152361-gh/fadara01/1/head 2025-12-04T09:21:04.5662682Z * [new branch] revert-156870-gh/skarjala/3/head -> origin/revert-156870-gh/skarjala/3/head 2025-12-04T09:21:04.5664474Z * [new branch] revert-157914-cherry-pick-157503-by-pytorch_bot_bot_ -> origin/revert-157914-cherry-pick-157503-by-pytorch_bot_bot_ 2025-12-04T09:21:04.5665648Z * [new branch] revert-hoo-invoke-subgraph -> origin/revert-hoo-invoke-subgraph 2025-12-04T09:21:04.5667196Z * [new branch] revert_always_build_distributed -> origin/revert_always_build_distributed 2025-12-04T09:21:04.5668533Z * [new branch] rms_norm_patch -> origin/rms_norm_patch 2025-12-04T09:21:04.5670374Z * [new branch] ruisi/fix_all_to_all_estimation -> origin/ruisi/fix_all_to_all_estimation 2025-12-04T09:21:04.5671514Z * [new branch] ruisi/fix_comm_estimation -> origin/ruisi/fix_comm_estimation 2025-12-04T09:21:04.5672750Z * [new branch] ruisi/fix_dynamic_shape_estimation -> origin/ruisi/fix_dynamic_shape_estimation 2025-12-04T09:21:04.5674314Z * [new branch] ruisi/fix_llama3_autobucketing -> origin/ruisi/fix_llama3_autobucketing 2025-12-04T09:21:04.5675847Z * [new branch] ruisi/fix_manual_bucketing_ep_pass -> origin/ruisi/fix_manual_bucketing_ep_pass 2025-12-04T09:21:04.5677327Z * [new branch] ruisi/manual_bucket_pass -> origin/ruisi/manual_bucket_pass 2025-12-04T09:21:04.5679417Z * [new branch] ryanguo99/cleanup-dynamo-expected-failures -> origin/ryanguo99/cleanup-dynamo-expected-failures 2025-12-04T09:21:04.5680728Z * [new branch] ryanguo99/fix-closure-var -> origin/ryanguo99/fix-closure-var 2025-12-04T09:21:04.5682380Z * [new branch] rzou/faketensor_bench -> origin/rzou/faketensor_bench 2025-12-04T09:21:04.5683444Z * [new branch] rzou/njt -> origin/rzou/njt 2025-12-04T09:21:04.5684690Z * [new branch] rzou/pca -> origin/rzou/pca 2025-12-04T09:21:04.5685817Z * [new branch] rzou/realprop -> origin/rzou/realprop 2025-12-04T09:21:04.5687148Z * [new branch] samplevllm -> origin/samplevllm 2025-12-04T09:21:04.5689271Z * [new branch] sanchitintel/weird_thing_with_test_cpu_select_algorithm -> origin/sanchitintel/weird_thing_with_test_cpu_select_algorithm 2025-12-04T09:21:04.5690419Z * [new branch] sapling-pr-archive-SS-JIA -> origin/sapling-pr-archive-SS-JIA 2025-12-04T09:21:04.5691847Z * [new branch] sapling-pr-archive-tushar00jain -> origin/sapling-pr-archive-tushar00jain 2025-12-04T09:21:04.5692950Z * [new branch] save -> origin/save 2025-12-04T09:21:04.5694313Z * [new branch] scaled_mm -> origin/scaled_mm 2025-12-04T09:21:04.5695577Z * [new branch] scan_attempt -> origin/scan_attempt 2025-12-04T09:21:04.5697161Z * [new branch] sdym/2.5.1 -> origin/sdym/2.5.1 2025-12-04T09:21:04.5698528Z * [new branch] sekyondaMeta-dynamoconfig-fix -> origin/sekyondaMeta-dynamoconfig-fix 2025-12-04T09:21:04.5700061Z * [new branch] shengf/fx-xform-perf -> origin/shengf/fx-xform-perf 2025-12-04T09:21:04.5702523Z * [new branch] shoumikhin-patch-1 -> origin/shoumikhin-patch-1 2025-12-04T09:21:04.5703592Z * [new branch] solve-accuracy-fix -> origin/solve-accuracy-fix 2025-12-04T09:21:04.5704880Z * [new branch] some_rocm_inductor_skips -> origin/some_rocm_inductor_skips 2025-12-04T09:21:04.5706672Z * [new branch] soulitzer/stash-tls-ac -> origin/soulitzer/stash-tls-ac 2025-12-04T09:21:04.5708148Z * [new branch] sparse-mm-bf16-support -> origin/sparse-mm-bf16-support 2025-12-04T09:21:04.5709476Z * [new branch] starterTaskUpdate -> origin/starterTaskUpdate 2025-12-04T09:21:04.5710765Z * [new branch] suo -> origin/suo 2025-12-04T09:21:04.5716885Z * [new branch] sve-poc -> origin/sve-poc 2025-12-04T09:21:04.5717269Z * [new branch] switch-bn -> origin/switch-bn 2025-12-04T09:21:04.5717749Z * [new branch] sy_annotation_in_autograd_hop -> origin/sy_annotation_in_autograd_hop 2025-12-04T09:21:04.5718004Z * [new branch] sy_aot_eager_record -> origin/sy_aot_eager_record 2025-12-04T09:21:04.5718255Z * [new branch] sy_custom_bucketing -> origin/sy_custom_bucketing 2025-12-04T09:21:04.5719429Z * [new branch] sy_debug_mode_test -> origin/sy_debug_mode_test 2025-12-04T09:21:04.5720606Z * [new branch] sy_deserialize -> origin/sy_deserialize 2025-12-04T09:21:04.5721883Z * [new branch] sy_dump_gm_code -> origin/sy_dump_gm_code 2025-12-04T09:21:04.5723176Z * [new branch] sy_exp -> origin/sy_exp 2025-12-04T09:21:04.5724523Z * [new branch] sy_export_annotation -> origin/sy_export_annotation 2025-12-04T09:21:04.5725796Z * [new branch] sy_invoke_subgraph -> origin/sy_invoke_subgraph 2025-12-04T09:21:04.5727040Z * [new branch] sy_kernel_bw_name -> origin/sy_kernel_bw_name 2025-12-04T09:21:04.5728857Z * [new branch] sy_multi_arch -> origin/sy_multi_arch 2025-12-04T09:21:04.5730124Z * [new branch] sy_nn_module_stack -> origin/sy_nn_module_stack 2025-12-04T09:21:04.5731401Z * [new branch] sy_original_dtensor -> origin/sy_original_dtensor 2025-12-04T09:21:04.5732614Z * [new branch] sy_profiler_cia -> origin/sy_profiler_cia 2025-12-04T09:21:04.5733863Z * [new branch] symm_mem_sync -> origin/symm_mem_sync 2025-12-04T09:21:04.5735286Z * [new branch] sympy-bottleneck-repro -> origin/sympy-bottleneck-repro 2025-12-04T09:21:04.5736583Z * [new branch] tensordict_integration -> origin/tensordict_integration 2025-12-04T09:21:04.5737977Z * [new branch] test-move-conda-builds -> origin/test-move-conda-builds 2025-12-04T09:21:04.5739611Z * [new branch] test-old -> origin/test-old 2025-12-04T09:21:04.5741255Z * [new branch] test/bmm_heur -> origin/test/bmm_heur 2025-12-04T09:21:04.5742946Z * [new branch] tianren/customOp_autotune_fix -> origin/tianren/customOp_autotune_fix 2025-12-04T09:21:04.5744194Z * [new branch] tianren/customOp_enable_max_autotune -> origin/tianren/customOp_enable_max_autotune 2025-12-04T09:21:04.5745255Z * [new branch] tianren/customOp_fusion -> origin/tianren/customOp_fusion 2025-12-04T09:21:04.5746515Z * [new branch] tianren/customop_collectiveop_benchmark -> origin/tianren/customop_collectiveop_benchmark 2025-12-04T09:21:04.5748402Z * [new branch] tianren/customop_collectiveop_benchmark_fix -> origin/tianren/customop_collectiveop_benchmark_fix 2025-12-04T09:21:04.5749817Z * [new branch] tianren/customop_dynamic_config -> origin/tianren/customop_dynamic_config 2025-12-04T09:21:04.5751055Z * [new branch] tianren/dynamic_range_input -> origin/tianren/dynamic_range_input 2025-12-04T09:21:04.5752385Z * [new branch] tianren/dynamic_range_input_fix -> origin/tianren/dynamic_range_input_fix 2025-12-04T09:21:04.5753696Z * [new branch] tianren/dynamic_range_input_merge -> origin/tianren/dynamic_range_input_merge 2025-12-04T09:21:04.5755005Z * [new branch] tianren/flex_paged_attn_fix_temp -> origin/tianren/flex_paged_attn_fix_temp 2025-12-04T09:21:04.5756227Z * [new branch] tianren/fx_codegen_dump -> origin/tianren/fx_codegen_dump 2025-12-04T09:21:04.5757489Z * [new branch] tianren/symmetric_memory -> origin/tianren/symmetric_memory 2025-12-04T09:21:04.5758619Z * [new branch] tianren/test -> origin/tianren/test 2025-12-04T09:21:04.5760087Z * [new branch] tidy_performance_cyy -> origin/tidy_performance_cyy 2025-12-04T09:21:04.5761285Z * [new branch] tmp -> origin/tmp 2025-12-04T09:21:04.5762595Z * [new branch] torchtitan_ep -> origin/torchtitan_ep 2025-12-04T09:21:04.5763911Z * [new branch] torchtitan_integration -> origin/torchtitan_integration 2025-12-04T09:21:04.5765275Z * [new branch] trace_fsdp_torchtune_lora -> origin/trace_fsdp_torchtune_lora 2025-12-04T09:21:04.5766468Z * [new branch] traceable_fsdp_unit_tests -> origin/traceable_fsdp_unit_tests 2025-12-04T09:21:04.5767711Z * [new branch] tree_loop_vec_base -> origin/tree_loop_vec_base 2025-12-04T09:21:04.5768976Z * [new branch] triton_kernel -> origin/triton_kernel 2025-12-04T09:21:04.5770227Z * [new branch] tt_pkg_1908 -> origin/tt_pkg_1908 2025-12-04T09:21:04.5771496Z * [new branch] type_dec -> origin/type_dec 2025-12-04T09:21:04.5772839Z * [new branch] udate-sphinx-dependancies -> origin/udate-sphinx-dependancies 2025-12-04T09:21:04.5779636Z * [new branch] update-audio-commit-hash/17630256502-1803-1 -> origin/update-audio-commit-hash/17630256502-1803-1 2025-12-04T09:21:04.5785622Z * [new branch] update-audio-commit-hash/19087141161-1916-1 -> origin/update-audio-commit-hash/19087141161-1916-1 2025-12-04T09:21:04.5786923Z * [new branch] update-audio-commit-hash/19250643381-1929-1 -> origin/update-audio-commit-hash/19250643381-1929-1 2025-12-04T09:21:04.5788162Z * [new branch] update-audio-commit-hash/19397724337-1935-1 -> origin/update-audio-commit-hash/19397724337-1935-1 2025-12-04T09:21:04.5789367Z * [new branch] update-audio-commit-hash/19555670148-1941-1 -> origin/update-audio-commit-hash/19555670148-1941-1 2025-12-04T09:21:04.5790811Z * [new branch] update-audio-commit-hash/19750627930-1946-1 -> origin/update-audio-commit-hash/19750627930-1946-1 2025-12-04T09:21:04.5792826Z * [new branch] update-triton-commit-hash/13663274526-1487-2 -> origin/update-triton-commit-hash/13663274526-1487-2 2025-12-04T09:21:04.5794528Z * [new branch] update-vision-commit-hash/19087141161-1916-1 -> origin/update-vision-commit-hash/19087141161-1916-1 2025-12-04T09:21:04.5795737Z * [new branch] update-vision-commit-hash/19184897099-1925-1 -> origin/update-vision-commit-hash/19184897099-1925-1 2025-12-04T09:21:04.5796750Z * [new branch] update-vision-commit-hash/19250643381-1929-1 -> origin/update-vision-commit-hash/19250643381-1929-1 2025-12-04T09:21:04.5798048Z * [new branch] update-vision-commit-hash/19381328640-1934-1 -> origin/update-vision-commit-hash/19381328640-1934-1 2025-12-04T09:21:04.5799229Z * [new branch] update-vision-commit-hash/19485237164-1938-1 -> origin/update-vision-commit-hash/19485237164-1938-1 2025-12-04T09:21:04.5801605Z * [new branch] update-vllm-commit-hash/18451675449-1879-1 -> origin/update-vllm-commit-hash/18451675449-1879-1 2025-12-04T09:21:04.5803132Z * [new branch] update-vllm-dockerfile -> origin/update-vllm-dockerfile 2025-12-04T09:21:04.5804922Z * [new branch] update-xla-commit-hash/19224287370-211-1 -> origin/update-xla-commit-hash/19224287370-211-1 2025-12-04T09:21:04.5806147Z * [new branch] update-xla-commit-hash/19422028566-212-1 -> origin/update-xla-commit-hash/19422028566-212-1 2025-12-04T09:21:04.5807340Z * [new branch] update-xla-commit-hash/19626841311-213-1 -> origin/update-xla-commit-hash/19626841311-213-1 2025-12-04T09:21:04.5808683Z * [new branch] update_docs_torch_multinomial_issue#125388 -> origin/update_docs_torch_multinomial_issue#125388 2025-12-04T09:21:04.5809810Z * [new branch] update_operator_readme -> origin/update_operator_readme 2025-12-04T09:21:04.5811213Z * [new branch] update_slow_tests_1722488736 -> origin/update_slow_tests_1722488736 2025-12-04T09:21:04.5812516Z * [new branch] update_slow_tests_1722879173 -> origin/update_slow_tests_1722879173 2025-12-04T09:21:04.5813780Z * [new branch] update_slow_tests_1762155677 -> origin/update_slow_tests_1762155677 2025-12-04T09:21:04.5815298Z * [new branch] update_slow_tests_1763365283 -> origin/update_slow_tests_1763365283 2025-12-04T09:21:04.5816481Z * [new branch] update_submodule_FBGEMM -> origin/update_submodule_FBGEMM 2025-12-04T09:21:04.5817768Z * [new branch] update_submodule_kineto -> origin/update_submodule_kineto 2025-12-04T09:21:04.5819034Z * [new branch] update_submodule_tensorpipe -> origin/update_submodule_tensorpipe 2025-12-04T09:21:04.5820303Z * [new branch] upload-tests-for-autorevert -> origin/upload-tests-for-autorevert 2025-12-04T09:21:04.5821595Z * [new branch] v0.1.2 -> origin/v0.1.2 2025-12-04T09:21:04.5823173Z * [new branch] v1.0.1 -> origin/v1.0.1 2025-12-04T09:21:04.5824537Z * [new branch] v1.0.3 -> origin/v1.0.3 2025-12-04T09:21:04.5826035Z * [new branch] v1.1.0 -> origin/v1.1.0 2025-12-04T09:21:04.5827864Z * [new branch] v1.2.0 -> origin/v1.2.0 2025-12-04T09:21:04.5829120Z * [new branch] v1.3.0 -> origin/v1.3.0 2025-12-04T09:21:04.5830516Z * [new branch] v1.3.1 -> origin/v1.3.1 2025-12-04T09:21:04.5831852Z * [new branch] validate_fn -> origin/validate_fn 2025-12-04T09:21:04.5833326Z * [new branch] validations_2.6 -> origin/validations_2.6 2025-12-04T09:21:04.5834755Z * [new branch] validations_2.8 -> origin/validations_2.8 2025-12-04T09:21:04.5836062Z * [new branch] varlen-api -> origin/varlen-api 2025-12-04T09:21:04.5841883Z * [new branch] varlen-api-backup -> origin/varlen-api-backup 2025-12-04T09:21:04.5842196Z * [new branch] varlen_batch_invariance -> origin/varlen_batch_invariance 2025-12-04T09:21:04.5842425Z * [new branch] viable/strict -> origin/viable/strict 2025-12-04T09:21:04.5842855Z * [new branch] vishal9-team/dtensor_parallelism_toy -> origin/vishal9-team/dtensor_parallelism_toy 2025-12-04T09:21:04.5843915Z * [new branch] vllmbuildci -> origin/vllmbuildci 2025-12-04T09:21:04.5845171Z * [new branch] vllmpin -> origin/vllmpin 2025-12-04T09:21:04.5846693Z * [new branch] vscode-recommend-pyrefly -> origin/vscode-recommend-pyrefly 2025-12-04T09:21:04.5848055Z * [new branch] wdvr-patch-1 -> origin/wdvr-patch-1 2025-12-04T09:21:04.5849813Z * [new branch] wdvr/iss_145259 -> origin/wdvr/iss_145259 2025-12-04T09:21:04.5851397Z * [new branch] whc/pei -> origin/whc/pei 2025-12-04T09:21:04.5852586Z * [new branch] whc/pp_fix -> origin/whc/pp_fix 2025-12-04T09:21:04.5853938Z * [new branch] whc/sharding -> origin/whc/sharding 2025-12-04T09:21:04.5855395Z * [new branch] whc/sharding2 -> origin/whc/sharding2 2025-12-04T09:21:04.5856086Z * [new branch] whc/uneven -> origin/whc/uneven 2025-12-04T09:21:04.5857693Z * [new branch] whc/uneven-merge -> origin/whc/uneven-merge 2025-12-04T09:21:04.5858963Z * [new branch] win_warnings -> origin/win_warnings 2025-12-04T09:21:04.5860193Z * [new branch] windows_libtorch_free -> origin/windows_libtorch_free 2025-12-04T09:21:04.5861436Z * [new branch] xmfan-war -> origin/xmfan-war 2025-12-04T09:21:04.5863218Z * [new branch] xmfan/ca_0516 -> origin/xmfan/ca_0516 2025-12-04T09:21:04.5864450Z * [new branch] xmfan/ca_1051b93192 -> origin/xmfan/ca_1051b93192 2025-12-04T09:21:04.5865795Z * [new branch] xmfan/ca_1a722f62c248391fc4a542e8851a5559aa356ae8 -> origin/xmfan/ca_1a722f62c248391fc4a542e8851a5559aa356ae8 2025-12-04T09:21:04.5866657Z * [new branch] xmfan/ca_5a2be192d1 -> origin/xmfan/ca_5a2be192d1 2025-12-04T09:21:04.5868113Z * [new branch] xmfan/ca_9d59b516e9 -> origin/xmfan/ca_9d59b516e9 2025-12-04T09:21:04.5869292Z * [new branch] xmfan/ca_apr8 -> origin/xmfan/ca_apr8 2025-12-04T09:21:04.5870494Z * [new branch] xmfan/ca_base -> origin/xmfan/ca_base 2025-12-04T09:21:04.5871969Z * [new branch] xmfan/ca_dynamic -> origin/xmfan/ca_dynamic 2025-12-04T09:21:04.5873543Z * [new branch] xmfan/ca_fix_dyn -> origin/xmfan/ca_fix_dyn 2025-12-04T09:21:04.5874846Z * [new branch] xmfan/ca_fix_lowering -> origin/xmfan/ca_fix_lowering 2025-12-04T09:21:04.5876043Z * [new branch] xmfan/ca_fix_polyfills -> origin/xmfan/ca_fix_polyfills 2025-12-04T09:21:04.5877110Z * [new branch] xmfan/ca_jan3 -> origin/xmfan/ca_jan3 2025-12-04T09:21:04.5878359Z * [new branch] xmfan/ca_jun18 -> origin/xmfan/ca_jun18 2025-12-04T09:21:04.5879714Z * [new branch] xmfan/ca_jun24 -> origin/xmfan/ca_jun24 2025-12-04T09:21:04.5880940Z * [new branch] xmfan/ca_nested -> origin/xmfan/ca_nested 2025-12-04T09:21:04.5882292Z * [new branch] xmfan/ca_overhead -> origin/xmfan/ca_overhead 2025-12-04T09:21:04.5883531Z * [new branch] xmfan/ca_overhead_0eba7e5451 -> origin/xmfan/ca_overhead_0eba7e5451 2025-12-04T09:21:04.5884702Z * [new branch] xmfan/cacu_jun18 -> origin/xmfan/cacu_jun18 2025-12-04T09:21:04.5888332Z * [new branch] xmfan/cacu_jun19 -> origin/xmfan/cacu_jun19 2025-12-04T09:21:04.5888599Z * [new branch] xmfan/cacu_jun4 -> origin/xmfan/cacu_jun4 2025-12-04T09:21:04.5888927Z * [new branch] xmfan/disable_duck_shape -> origin/xmfan/disable_duck_shape 2025-12-04T09:21:04.5889580Z * [new branch] xmfan/fca_cpp_node_passthrough -> origin/xmfan/fca_cpp_node_passthrough 2025-12-04T09:21:04.5891300Z * [new branch] xmfan/post_3945954741e2d37023c5d6954f9483008e0892f9 -> origin/xmfan/post_3945954741e2d37023c5d6954f9483008e0892f9 2025-12-04T09:21:04.5892282Z * [new branch] xmfan/pre_3945954741e2d37023c5d6954f9483008e0892f9 -> origin/xmfan/pre_3945954741e2d37023c5d6954f9483008e0892f9 2025-12-04T09:21:04.5893286Z * [new branch] xmfan/single_step -> origin/xmfan/single_step 2025-12-04T09:21:04.5894484Z * [new branch] xmfan/sth_0829 -> origin/xmfan/sth_0829 2025-12-04T09:21:04.5895769Z * [new branch] xmfan/test -> origin/xmfan/test 2025-12-04T09:21:04.5897511Z * [new branch] yguo/debug-0226-constexpr -> origin/yguo/debug-0226-constexpr 2025-12-04T09:21:04.5898651Z * [new branch] yguo/new_latest_changes -> origin/yguo/new_latest_changes 2025-12-04T09:21:04.5900286Z * [new branch] yguo/patch_constexpr_changes -> origin/yguo/patch_constexpr_changes 2025-12-04T09:21:04.5905362Z * [new branch] yiming/bootcamp -> origin/yiming/bootcamp 2025-12-04T09:21:04.5906798Z * [new branch] yiming/run_with_start_end_rng_hop -> origin/yiming/run_with_start_end_rng_hop 2025-12-04T09:21:04.5908113Z * [new branch] yolo-llama3 -> origin/yolo-llama3 2025-12-04T09:21:04.5910250Z * [new branch] zainr/canary-test -> origin/zainr/canary-test 2025-12-04T09:21:04.5911688Z * [new branch] zainr/cleanup-gh-runners -> origin/zainr/cleanup-gh-runners 2025-12-04T09:21:04.5912780Z * [new branch] zainr/pull-migration-c -> origin/zainr/pull-migration-c 2025-12-04T09:21:04.5913859Z * [new branch] zainr/test2 -> origin/zainr/test2 2025-12-04T09:21:04.5915435Z * [new branch] zasdfgbnm-patch-3 -> origin/zasdfgbnm-patch-3 2025-12-04T09:21:04.5916620Z * [new branch] zb2p -> origin/zb2p 2025-12-04T09:21:04.5917989Z * [new branch] zeros-and-scatter-part2 -> origin/zeros-and-scatter-part2 2025-12-04T09:21:04.5920027Z * [new branch] zhxchen17/ci/vllm_lora_oom -> origin/zhxchen17/ci/vllm_lora_oom 2025-12-04T09:21:04.5921279Z * [new branch] zhxchen17/ci/vllm_multimodal_oom -> origin/zhxchen17/ci/vllm_multimodal_oom 2025-12-04T09:21:04.5922909Z * [new branch] zhxchen17/ci/vllm_pin -> origin/zhxchen17/ci/vllm_pin 2025-12-04T09:21:04.5924749Z * [new branch] zhxchen17/dynamo/unsafe_drop_all_guards -> origin/zhxchen17/dynamo/unsafe_drop_all_guards 2025-12-04T09:21:04.5926188Z * [new branch] zhxchen17/export/call_override -> origin/zhxchen17/export/call_override 2025-12-04T09:21:04.5927314Z * [new branch] zhxchen17/export/codemod1 -> origin/zhxchen17/export/codemod1 2025-12-04T09:21:04.5928601Z * [new branch] zhxchen17/export/ctx_return -> origin/zhxchen17/export/ctx_return 2025-12-04T09:21:04.5929943Z * [new branch] zhxchen17/export/disable_side_effect_warn -> origin/zhxchen17/export/disable_side_effect_warn 2025-12-04T09:21:04.5931015Z * [new branch] zhxchen17/export/pytree_check -> origin/zhxchen17/export/pytree_check 2025-12-04T09:21:04.5932545Z * [new branch] zhxchen17/precompile/aoti -> origin/zhxchen17/precompile/aoti 2025-12-04T09:21:04.5933792Z * [new branch] zhxchen17/precompile/globals -> origin/zhxchen17/precompile/globals 2025-12-04T09:21:04.5935453Z * [new branch] zhxchen17/precompile/inductor_guards -> origin/zhxchen17/precompile/inductor_guards 2025-12-04T09:21:04.5937247Z * [new branch] zhxchen17/scratch/0 -> origin/zhxchen17/scratch/0 2025-12-04T09:21:04.5938627Z * [new branch] zhxchen17/torch_export_api_update -> origin/zhxchen17/torch_export_api_update 2025-12-04T09:21:04.5940309Z * [new branch] zhxhcen17/moodycamel -> origin/zhxhcen17/moodycamel 2025-12-04T09:21:04.5942001Z * [new branch] zxiiro/build-times -> origin/zxiiro/build-times 2025-12-04T09:21:04.5943676Z * [new branch] zxiiro/c7i.2xlarge -> origin/zxiiro/c7i.2xlarge 2025-12-04T09:21:04.5944952Z * [new branch] zxiiro/c7i.2xlarge.h100 -> origin/zxiiro/c7i.2xlarge.h100 2025-12-04T09:21:04.5946131Z * [new branch] zxiiro/main -> origin/zxiiro/main 2025-12-04T09:21:04.5947684Z * [new branch] zxiiro/risc64 -> origin/zxiiro/risc64 2025-12-04T09:21:04.5949014Z * [new branch] zxiiro/test-multicloud-arc -> origin/zxiiro/test-multicloud-arc 2025-12-04T09:21:04.5950384Z * [new tag] bc2caa7fdf006894eff7af936babde69ab5a40f8-huydhn-debug -> bc2caa7fdf006894eff7af936babde69ab5a40f8-huydhn-debug 2025-12-04T09:21:04.5951128Z * [new tag] ci/binaries/77164 -> ci/binaries/77164 2025-12-04T09:21:04.5952236Z * [new tag] ciflow/b200/115316 -> ciflow/b200/115316 2025-12-04T09:21:04.5953512Z * [new tag] ciflow/b200/160685 -> ciflow/b200/160685 2025-12-04T09:21:04.5954260Z * [new tag] ciflow/b200/161607 -> ciflow/b200/161607 2025-12-04T09:21:04.5955007Z * [new tag] ciflow/b200/161938 -> ciflow/b200/161938 2025-12-04T09:21:04.5956047Z * [new tag] ciflow/b200/167207 -> ciflow/b200/167207 2025-12-04T09:21:04.5956758Z * [new tag] ciflow/b200/167989 -> ciflow/b200/167989 2025-12-04T09:21:04.5957752Z * [new tag] ciflow/b200/168096 -> ciflow/b200/168096 2025-12-04T09:21:04.5958569Z * [new tag] ciflow/b200/168175 -> ciflow/b200/168175 2025-12-04T09:21:04.5959707Z * [new tag] ciflow/b200/168195 -> ciflow/b200/168195 2025-12-04T09:21:04.5960463Z * [new tag] ciflow/b200/169200 -> ciflow/b200/169200 2025-12-04T09:21:04.5961428Z * [new tag] ciflow/b200/169216 -> ciflow/b200/169216 2025-12-04T09:21:04.5963086Z * [new tag] ciflow/b200/169380 -> ciflow/b200/169380 2025-12-04T09:21:04.5967814Z * [new tag] ciflow/b200/169412 -> ciflow/b200/169412 2025-12-04T09:21:04.5968043Z * [new tag] ciflow/b200/169470 -> ciflow/b200/169470 2025-12-04T09:21:04.5968269Z * [new tag] ciflow/b200/169471 -> ciflow/b200/169471 2025-12-04T09:21:04.5968464Z * [new tag] ciflow/b200/169472 -> ciflow/b200/169472 2025-12-04T09:21:04.5968682Z * [new tag] ciflow/b200/169514 -> ciflow/b200/169514 2025-12-04T09:21:04.5968903Z * [new tag] ciflow/b200/169517 -> ciflow/b200/169517 2025-12-04T09:21:04.5970110Z * [new tag] ciflow/binaries/165922 -> ciflow/binaries/165922 2025-12-04T09:21:04.5970884Z * [new tag] ciflow/binaries/169510 -> ciflow/binaries/169510 2025-12-04T09:21:04.5972039Z * [new tag] ciflow/binaries_wheel/157994 -> ciflow/binaries_wheel/157994 2025-12-04T09:21:04.5972895Z * [new tag] ciflow/binaries_wheel/166829 -> ciflow/binaries_wheel/166829 2025-12-04T09:21:04.5973689Z * [new tag] ciflow/binaries_wheel/167972 -> ciflow/binaries_wheel/167972 2025-12-04T09:21:04.5974806Z * [new tag] ciflow/binaries_wheel/167981 -> ciflow/binaries_wheel/167981 2025-12-04T09:21:04.5975521Z * [new tag] ciflow/dynamo/167695 -> ciflow/dynamo/167695 2025-12-04T09:21:04.5976331Z * [new tag] ciflow/dynamo/168096 -> ciflow/dynamo/168096 2025-12-04T09:21:04.5977327Z * [new tag] ciflow/dynamo/169525 -> ciflow/dynamo/169525 2025-12-04T09:21:04.5978321Z * [new tag] ciflow/h100-cutlass-backend/161938 -> ciflow/h100-cutlass-backend/161938 2025-12-04T09:21:04.5979225Z * [new tag] ciflow/h100-cutlass-backend/161940 -> ciflow/h100-cutlass-backend/161940 2025-12-04T09:21:04.5980266Z * [new tag] ciflow/h100-distributed/168923 -> ciflow/h100-distributed/168923 2025-12-04T09:21:04.5981079Z * [new tag] ciflow/h100-symm-mem/167552 -> ciflow/h100-symm-mem/167552 2025-12-04T09:21:04.5981820Z * [new tag] ciflow/h100-symm-mem/168129 -> ciflow/h100-symm-mem/168129 2025-12-04T09:21:04.5982559Z * [new tag] ciflow/h100-symm-mem/168917 -> ciflow/h100-symm-mem/168917 2025-12-04T09:21:04.5983718Z * [new tag] ciflow/h100-symm-mem/169156 -> ciflow/h100-symm-mem/169156 2025-12-04T09:21:04.5984542Z * [new tag] ciflow/h100-symm-mem/169200 -> ciflow/h100-symm-mem/169200 2025-12-04T09:21:04.5985308Z * [new tag] ciflow/h100-symm-mem/169216 -> ciflow/h100-symm-mem/169216 2025-12-04T09:21:04.5986035Z * [new tag] ciflow/h100-symm-mem/169338 -> ciflow/h100-symm-mem/169338 2025-12-04T09:21:04.5987024Z * [new tag] ciflow/h100-symm-mem/169355 -> ciflow/h100-symm-mem/169355 2025-12-04T09:21:04.5988001Z * [new tag] ciflow/h100-symm-mem/169543 -> ciflow/h100-symm-mem/169543 2025-12-04T09:21:04.5988967Z * [new tag] ciflow/h100/115316 -> ciflow/h100/115316 2025-12-04T09:21:04.5989688Z * [new tag] ciflow/h100/160685 -> ciflow/h100/160685 2025-12-04T09:21:04.5990457Z * [new tag] ciflow/h100/160729 -> ciflow/h100/160729 2025-12-04T09:21:04.5991251Z * [new tag] ciflow/h100/161607 -> ciflow/h100/161607 2025-12-04T09:21:04.5992012Z * [new tag] ciflow/h100/161938 -> ciflow/h100/161938 2025-12-04T09:21:04.5992854Z * [new tag] ciflow/h100/167207 -> ciflow/h100/167207 2025-12-04T09:21:04.5993548Z * [new tag] ciflow/h100/167989 -> ciflow/h100/167989 2025-12-04T09:21:04.5994303Z * [new tag] ciflow/h100/168096 -> ciflow/h100/168096 2025-12-04T09:21:04.5995099Z * [new tag] ciflow/h100/168175 -> ciflow/h100/168175 2025-12-04T09:21:04.5996357Z * [new tag] ciflow/h100/168195 -> ciflow/h100/168195 2025-12-04T09:21:04.5997073Z * [new tag] ciflow/h100/168980 -> ciflow/h100/168980 2025-12-04T09:21:04.5998130Z * [new tag] ciflow/h100/169200 -> ciflow/h100/169200 2025-12-04T09:21:04.5999198Z * [new tag] ciflow/h100/169216 -> ciflow/h100/169216 2025-12-04T09:21:04.6000178Z * [new tag] ciflow/h100/169380 -> ciflow/h100/169380 2025-12-04T09:21:04.6001421Z * [new tag] ciflow/h100/169412 -> ciflow/h100/169412 2025-12-04T09:21:04.6002501Z * [new tag] ciflow/h100/169470 -> ciflow/h100/169470 2025-12-04T09:21:04.6003263Z * [new tag] ciflow/h100/169471 -> ciflow/h100/169471 2025-12-04T09:21:04.6004088Z * [new tag] ciflow/h100/169472 -> ciflow/h100/169472 2025-12-04T09:21:04.6004907Z * [new tag] ciflow/h100/169514 -> ciflow/h100/169514 2025-12-04T09:21:04.6005987Z * [new tag] ciflow/inductor-cu126/168096 -> ciflow/inductor-cu126/168096 2025-12-04T09:21:04.6007353Z * [new tag] ciflow/inductor-micro-benchmark-cpu-x86/168096 -> ciflow/inductor-micro-benchmark-cpu-x86/168096 2025-12-04T09:21:04.6008169Z * [new tag] ciflow/inductor-micro-benchmark/166165 -> ciflow/inductor-micro-benchmark/166165 2025-12-04T09:21:04.6009025Z * [new tag] ciflow/inductor-micro-benchmark/168096 -> ciflow/inductor-micro-benchmark/168096 2025-12-04T09:21:04.6010197Z * [new tag] ciflow/inductor-perf-compare/168096 -> ciflow/inductor-perf-compare/168096 2025-12-04T09:21:04.6011534Z * [new tag] ciflow/inductor-perf-test-nightly-rocm-mi300/168073 -> ciflow/inductor-perf-test-nightly-rocm-mi300/168073 2025-12-04T09:21:04.6012235Z * [new tag] ciflow/inductor-perf-test-nightly-rocm-mi300/168096 -> ciflow/inductor-perf-test-nightly-rocm-mi300/168096 2025-12-04T09:21:04.6013332Z * [new tag] ciflow/inductor-perf-test-nightly-rocm-mi300/169024 -> ciflow/inductor-perf-test-nightly-rocm-mi300/169024 2025-12-04T09:21:04.6014230Z * [new tag] ciflow/inductor-perf-test-nightly-rocm-mi355/169024 -> ciflow/inductor-perf-test-nightly-rocm-mi355/169024 2025-12-04T09:21:04.6015051Z * [new tag] ciflow/inductor-perf-test-nightly/168096 -> ciflow/inductor-perf-test-nightly/168096 2025-12-04T09:21:04.6016193Z * [new tag] ciflow/inductor-periodic/168096 -> ciflow/inductor-periodic/168096 2025-12-04T09:21:04.6016909Z * [new tag] ciflow/inductor-periodic/169024 -> ciflow/inductor-periodic/169024 2025-12-04T09:21:04.6017834Z * [new tag] ciflow/inductor-periodic/169425 -> ciflow/inductor-periodic/169425 2025-12-04T09:21:04.6018944Z * [new tag] ciflow/inductor-rocm-mi200/165545 -> ciflow/inductor-rocm-mi200/165545 2025-12-04T09:21:04.6019779Z * [new tag] ciflow/inductor-rocm-mi200/165997 -> ciflow/inductor-rocm-mi200/165997 2025-12-04T09:21:04.6020516Z * [new tag] ciflow/inductor-rocm-mi200/168096 -> ciflow/inductor-rocm-mi200/168096 2025-12-04T09:21:04.6021392Z * [new tag] ciflow/inductor-rocm-mi200/169063 -> ciflow/inductor-rocm-mi200/169063 2025-12-04T09:21:04.6022142Z * [new tag] ciflow/inductor-rocm-mi200/169425 -> ciflow/inductor-rocm-mi200/169425 2025-12-04T09:21:04.6023151Z * [new tag] ciflow/inductor-rocm-mi300/165545 -> ciflow/inductor-rocm-mi300/165545 2025-12-04T09:21:04.6023825Z * [new tag] ciflow/inductor-rocm-mi300/168096 -> ciflow/inductor-rocm-mi300/168096 2025-12-04T09:21:04.6030078Z * [new tag] ciflow/inductor-rocm-mi300/169063 -> ciflow/inductor-rocm-mi300/169063 2025-12-04T09:21:04.6030781Z * [new tag] ciflow/inductor-rocm-mi300/169425 -> ciflow/inductor-rocm-mi300/169425 2025-12-04T09:21:04.6031967Z * [new tag] ciflow/inductor-rocm/162052 -> ciflow/inductor-rocm/162052 2025-12-04T09:21:04.6032725Z * [new tag] ciflow/inductor-rocm/168971 -> ciflow/inductor-rocm/168971 2025-12-04T09:21:04.6034201Z * [new tag] ciflow/inductor-windows/168096 -> ciflow/inductor-windows/168096 2025-12-04T09:21:04.6035065Z * [new tag] ciflow/inductor/144542 -> ciflow/inductor/144542 2025-12-04T09:21:04.6036085Z * [new tag] ciflow/inductor/146506 -> ciflow/inductor/146506 2025-12-04T09:21:04.6036728Z * [new tag] ciflow/inductor/147990 -> ciflow/inductor/147990 2025-12-04T09:21:04.6038296Z * [new tag] ciflow/inductor/148294 -> ciflow/inductor/148294 2025-12-04T09:21:04.6038982Z * [new tag] ciflow/inductor/148492 -> ciflow/inductor/148492 2025-12-04T09:21:04.6039762Z * [new tag] ciflow/inductor/157149 -> ciflow/inductor/157149 2025-12-04T09:21:04.6040496Z * [new tag] ciflow/inductor/157994 -> ciflow/inductor/157994 2025-12-04T09:21:04.6041254Z * [new tag] ciflow/inductor/160685 -> ciflow/inductor/160685 2025-12-04T09:21:04.6042025Z * [new tag] ciflow/inductor/160686 -> ciflow/inductor/160686 2025-12-04T09:21:04.6042789Z * [new tag] ciflow/inductor/160687 -> ciflow/inductor/160687 2025-12-04T09:21:04.6043521Z * [new tag] ciflow/inductor/160688 -> ciflow/inductor/160688 2025-12-04T09:21:04.6044592Z * [new tag] ciflow/inductor/160706 -> ciflow/inductor/160706 2025-12-04T09:21:04.6045698Z * [new tag] ciflow/inductor/160729 -> ciflow/inductor/160729 2025-12-04T09:21:04.6046676Z * [new tag] ciflow/inductor/161938 -> ciflow/inductor/161938 2025-12-04T09:21:04.6047409Z * [new tag] ciflow/inductor/161939 -> ciflow/inductor/161939 2025-12-04T09:21:04.6048210Z * [new tag] ciflow/inductor/161940 -> ciflow/inductor/161940 2025-12-04T09:21:04.6048990Z * [new tag] ciflow/inductor/162052 -> ciflow/inductor/162052 2025-12-04T09:21:04.6049875Z * [new tag] ciflow/inductor/162275 -> ciflow/inductor/162275 2025-12-04T09:21:04.6050635Z * [new tag] ciflow/inductor/162795 -> ciflow/inductor/162795 2025-12-04T09:21:04.6051708Z * [new tag] ciflow/inductor/163245 -> ciflow/inductor/163245 2025-12-04T09:21:04.6052472Z * [new tag] ciflow/inductor/163335 -> ciflow/inductor/163335 2025-12-04T09:21:04.6053277Z * [new tag] ciflow/inductor/163503 -> ciflow/inductor/163503 2025-12-04T09:21:04.6054104Z * [new tag] ciflow/inductor/163942 -> ciflow/inductor/163942 2025-12-04T09:21:04.6055114Z * [new tag] ciflow/inductor/165270 -> ciflow/inductor/165270 2025-12-04T09:21:04.6055824Z * [new tag] ciflow/inductor/165274 -> ciflow/inductor/165274 2025-12-04T09:21:04.6056629Z * [new tag] ciflow/inductor/165322 -> ciflow/inductor/165322 2025-12-04T09:21:04.6057411Z * [new tag] ciflow/inductor/165597 -> ciflow/inductor/165597 2025-12-04T09:21:04.6058218Z * [new tag] ciflow/inductor/166063 -> ciflow/inductor/166063 2025-12-04T09:21:04.6059015Z * [new tag] ciflow/inductor/166075 -> ciflow/inductor/166075 2025-12-04T09:21:04.6059916Z * [new tag] ciflow/inductor/166165 -> ciflow/inductor/166165 2025-12-04T09:21:04.6060910Z * [new tag] ciflow/inductor/166254 -> ciflow/inductor/166254 2025-12-04T09:21:04.6061650Z * [new tag] ciflow/inductor/166483 -> ciflow/inductor/166483 2025-12-04T09:21:04.6062446Z * [new tag] ciflow/inductor/166494 -> ciflow/inductor/166494 2025-12-04T09:21:04.6063429Z * [new tag] ciflow/inductor/166545 -> ciflow/inductor/166545 2025-12-04T09:21:04.6064157Z * [new tag] ciflow/inductor/166788 -> ciflow/inductor/166788 2025-12-04T09:21:04.6065160Z * [new tag] ciflow/inductor/166846 -> ciflow/inductor/166846 2025-12-04T09:21:04.6065907Z * [new tag] ciflow/inductor/167300 -> ciflow/inductor/167300 2025-12-04T09:21:04.6066796Z * [new tag] ciflow/inductor/167407 -> ciflow/inductor/167407 2025-12-04T09:21:04.6068107Z * [new tag] ciflow/inductor/167536 -> ciflow/inductor/167536 2025-12-04T09:21:04.6068862Z * [new tag] ciflow/inductor/167552 -> ciflow/inductor/167552 2025-12-04T09:21:04.6069687Z * [new tag] ciflow/inductor/167555 -> ciflow/inductor/167555 2025-12-04T09:21:04.6070728Z * [new tag] ciflow/inductor/167583 -> ciflow/inductor/167583 2025-12-04T09:21:04.6071512Z * [new tag] ciflow/inductor/167599 -> ciflow/inductor/167599 2025-12-04T09:21:04.6072359Z * [new tag] ciflow/inductor/167647 -> ciflow/inductor/167647 2025-12-04T09:21:04.6073179Z * [new tag] ciflow/inductor/167677 -> ciflow/inductor/167677 2025-12-04T09:21:04.6074011Z * [new tag] ciflow/inductor/167680 -> ciflow/inductor/167680 2025-12-04T09:21:04.6074846Z * [new tag] ciflow/inductor/167695 -> ciflow/inductor/167695 2025-12-04T09:21:04.6075654Z * [new tag] ciflow/inductor/167742 -> ciflow/inductor/167742 2025-12-04T09:21:04.6076496Z * [new tag] ciflow/inductor/167768 -> ciflow/inductor/167768 2025-12-04T09:21:04.6077613Z * [new tag] ciflow/inductor/167773 -> ciflow/inductor/167773 2025-12-04T09:21:04.6078435Z * [new tag] ciflow/inductor/167781 -> ciflow/inductor/167781 2025-12-04T09:21:04.6079354Z * [new tag] ciflow/inductor/167880 -> ciflow/inductor/167880 2025-12-04T09:21:04.6080165Z * [new tag] ciflow/inductor/167887 -> ciflow/inductor/167887 2025-12-04T09:21:04.6080994Z * [new tag] ciflow/inductor/167972 -> ciflow/inductor/167972 2025-12-04T09:21:04.6081752Z * [new tag] ciflow/inductor/167989 -> ciflow/inductor/167989 2025-12-04T09:21:04.6082621Z * [new tag] ciflow/inductor/168002 -> ciflow/inductor/168002 2025-12-04T09:21:04.6083422Z * [new tag] ciflow/inductor/168050 -> ciflow/inductor/168050 2025-12-04T09:21:04.6084235Z * [new tag] ciflow/inductor/168051 -> ciflow/inductor/168051 2025-12-04T09:21:04.6085011Z * [new tag] ciflow/inductor/168052 -> ciflow/inductor/168052 2025-12-04T09:21:04.6085860Z * [new tag] ciflow/inductor/168073 -> ciflow/inductor/168073 2025-12-04T09:21:04.6091769Z * [new tag] ciflow/inductor/168096 -> ciflow/inductor/168096 2025-12-04T09:21:04.6092628Z * [new tag] ciflow/inductor/168114 -> ciflow/inductor/168114 2025-12-04T09:21:04.6093520Z * [new tag] ciflow/inductor/168115 -> ciflow/inductor/168115 2025-12-04T09:21:04.6094348Z * [new tag] ciflow/inductor/168127 -> ciflow/inductor/168127 2025-12-04T09:21:04.6095158Z * [new tag] ciflow/inductor/168129 -> ciflow/inductor/168129 2025-12-04T09:21:04.6096287Z * [new tag] ciflow/inductor/168157 -> ciflow/inductor/168157 2025-12-04T09:21:04.6097106Z * [new tag] ciflow/inductor/168175 -> ciflow/inductor/168175 2025-12-04T09:21:04.6098285Z * [new tag] ciflow/inductor/168185 -> ciflow/inductor/168185 2025-12-04T09:21:04.6099018Z * [new tag] ciflow/inductor/168195 -> ciflow/inductor/168195 2025-12-04T09:21:04.6099861Z * [new tag] ciflow/inductor/168209 -> ciflow/inductor/168209 2025-12-04T09:21:04.6100661Z * [new tag] ciflow/inductor/168266 -> ciflow/inductor/168266 2025-12-04T09:21:04.6103958Z * [new tag] ciflow/inductor/168316 -> ciflow/inductor/168316 2025-12-04T09:21:04.6105059Z * [new tag] ciflow/inductor/168326 -> ciflow/inductor/168326 2025-12-04T09:21:04.6105823Z * [new tag] ciflow/inductor/168368 -> ciflow/inductor/168368 2025-12-04T09:21:04.6106793Z * [new tag] ciflow/inductor/168894 -> ciflow/inductor/168894 2025-12-04T09:21:04.6107795Z * [new tag] ciflow/inductor/168934 -> ciflow/inductor/168934 2025-12-04T09:21:04.6108581Z * [new tag] ciflow/inductor/168939 -> ciflow/inductor/168939 2025-12-04T09:21:04.6109369Z * [new tag] ciflow/inductor/168946 -> ciflow/inductor/168946 2025-12-04T09:21:04.6110201Z * [new tag] ciflow/inductor/168950 -> ciflow/inductor/168950 2025-12-04T09:21:04.6111032Z * [new tag] ciflow/inductor/168951 -> ciflow/inductor/168951 2025-12-04T09:21:04.6111868Z * [new tag] ciflow/inductor/168952 -> ciflow/inductor/168952 2025-12-04T09:21:04.6112715Z * [new tag] ciflow/inductor/168955 -> ciflow/inductor/168955 2025-12-04T09:21:04.6113517Z * [new tag] ciflow/inductor/168971 -> ciflow/inductor/168971 2025-12-04T09:21:04.6114340Z * [new tag] ciflow/inductor/168979 -> ciflow/inductor/168979 2025-12-04T09:21:04.6115205Z * [new tag] ciflow/inductor/168980 -> ciflow/inductor/168980 2025-12-04T09:21:04.6116205Z * [new tag] ciflow/inductor/168983 -> ciflow/inductor/168983 2025-12-04T09:21:04.6116996Z * [new tag] ciflow/inductor/169006 -> ciflow/inductor/169006 2025-12-04T09:21:04.6117852Z * [new tag] ciflow/inductor/169023 -> ciflow/inductor/169023 2025-12-04T09:21:04.6118811Z * [new tag] ciflow/inductor/169024 -> ciflow/inductor/169024 2025-12-04T09:21:04.6119592Z * [new tag] ciflow/inductor/169025 -> ciflow/inductor/169025 2025-12-04T09:21:04.6120396Z * [new tag] ciflow/inductor/169066 -> ciflow/inductor/169066 2025-12-04T09:21:04.6121214Z * [new tag] ciflow/inductor/169091 -> ciflow/inductor/169091 2025-12-04T09:21:04.6122013Z * [new tag] ciflow/inductor/169102 -> ciflow/inductor/169102 2025-12-04T09:21:04.6122813Z * [new tag] ciflow/inductor/169103 -> ciflow/inductor/169103 2025-12-04T09:21:04.6123589Z * [new tag] ciflow/inductor/169121 -> ciflow/inductor/169121 2025-12-04T09:21:04.6124402Z * [new tag] ciflow/inductor/169134 -> ciflow/inductor/169134 2025-12-04T09:21:04.6125202Z * [new tag] ciflow/inductor/169135 -> ciflow/inductor/169135 2025-12-04T09:21:04.6126152Z * [new tag] ciflow/inductor/169141 -> ciflow/inductor/169141 2025-12-04T09:21:04.6126891Z * [new tag] ciflow/inductor/169151 -> ciflow/inductor/169151 2025-12-04T09:21:04.6127851Z * [new tag] ciflow/inductor/169161 -> ciflow/inductor/169161 2025-12-04T09:21:04.6128666Z * [new tag] ciflow/inductor/169167 -> ciflow/inductor/169167 2025-12-04T09:21:04.6129707Z * [new tag] ciflow/inductor/169177 -> ciflow/inductor/169177 2025-12-04T09:21:04.6130648Z * [new tag] ciflow/inductor/169185 -> ciflow/inductor/169185 2025-12-04T09:21:04.6131368Z * [new tag] ciflow/inductor/169196 -> ciflow/inductor/169196 2025-12-04T09:21:04.6132168Z * [new tag] ciflow/inductor/169200 -> ciflow/inductor/169200 2025-12-04T09:21:04.6132987Z * [new tag] ciflow/inductor/169204 -> ciflow/inductor/169204 2025-12-04T09:21:04.6133769Z * [new tag] ciflow/inductor/169216 -> ciflow/inductor/169216 2025-12-04T09:21:04.6134552Z * [new tag] ciflow/inductor/169219 -> ciflow/inductor/169219 2025-12-04T09:21:04.6135348Z * [new tag] ciflow/inductor/169220 -> ciflow/inductor/169220 2025-12-04T09:21:04.6136358Z * [new tag] ciflow/inductor/169230 -> ciflow/inductor/169230 2025-12-04T09:21:04.6137128Z * [new tag] ciflow/inductor/169242 -> ciflow/inductor/169242 2025-12-04T09:21:04.6138045Z * [new tag] ciflow/inductor/169245 -> ciflow/inductor/169245 2025-12-04T09:21:04.6139222Z * [new tag] ciflow/inductor/169260 -> ciflow/inductor/169260 2025-12-04T09:21:04.6139808Z * [new tag] ciflow/inductor/169282 -> ciflow/inductor/169282 2025-12-04T09:21:04.6140507Z * [new tag] ciflow/inductor/169286 -> ciflow/inductor/169286 2025-12-04T09:21:04.6141261Z * [new tag] ciflow/inductor/169299 -> ciflow/inductor/169299 2025-12-04T09:21:04.6142379Z * [new tag] ciflow/inductor/169304 -> ciflow/inductor/169304 2025-12-04T09:21:04.6144134Z * [new tag] ciflow/inductor/169305 -> ciflow/inductor/169305 2025-12-04T09:21:04.6145366Z * [new tag] ciflow/inductor/169308 -> ciflow/inductor/169308 2025-12-04T09:21:04.6146110Z * [new tag] ciflow/inductor/169319 -> ciflow/inductor/169319 2025-12-04T09:21:04.6147375Z * [new tag] ciflow/inductor/169326 -> ciflow/inductor/169326 2025-12-04T09:21:04.6148447Z * [new tag] ciflow/inductor/169332 -> ciflow/inductor/169332 2025-12-04T09:21:04.6149372Z * [new tag] ciflow/inductor/169333 -> ciflow/inductor/169333 2025-12-04T09:21:04.6154123Z * [new tag] ciflow/inductor/169336 -> ciflow/inductor/169336 2025-12-04T09:21:04.6154770Z * [new tag] ciflow/inductor/169340 -> ciflow/inductor/169340 2025-12-04T09:21:04.6155368Z * [new tag] ciflow/inductor/169341 -> ciflow/inductor/169341 2025-12-04T09:21:04.6155938Z * [new tag] ciflow/inductor/169343 -> ciflow/inductor/169343 2025-12-04T09:21:04.6156543Z * [new tag] ciflow/inductor/169346 -> ciflow/inductor/169346 2025-12-04T09:21:04.6157123Z * [new tag] ciflow/inductor/169348 -> ciflow/inductor/169348 2025-12-04T09:21:04.6157694Z * [new tag] ciflow/inductor/169350 -> ciflow/inductor/169350 2025-12-04T09:21:04.6158325Z * [new tag] ciflow/inductor/169355 -> ciflow/inductor/169355 2025-12-04T09:21:04.6159999Z * [new tag] ciflow/inductor/169370 -> ciflow/inductor/169370 2025-12-04T09:21:04.6161353Z * [new tag] ciflow/inductor/169375 -> ciflow/inductor/169375 2025-12-04T09:21:04.6162415Z * [new tag] ciflow/inductor/169389 -> ciflow/inductor/169389 2025-12-04T09:21:04.6163435Z * [new tag] ciflow/inductor/169391 -> ciflow/inductor/169391 2025-12-04T09:21:04.6164461Z * [new tag] ciflow/inductor/169393 -> ciflow/inductor/169393 2025-12-04T09:21:04.6165604Z * [new tag] ciflow/inductor/169399 -> ciflow/inductor/169399 2025-12-04T09:21:04.6166634Z * [new tag] ciflow/inductor/169400 -> ciflow/inductor/169400 2025-12-04T09:21:04.6167804Z * [new tag] ciflow/inductor/169415 -> ciflow/inductor/169415 2025-12-04T09:21:04.6168951Z * [new tag] ciflow/inductor/169417 -> ciflow/inductor/169417 2025-12-04T09:21:04.6169990Z * [new tag] ciflow/inductor/169418 -> ciflow/inductor/169418 2025-12-04T09:21:04.6171014Z * [new tag] ciflow/inductor/169430 -> ciflow/inductor/169430 2025-12-04T09:21:04.6172064Z * [new tag] ciflow/inductor/169432 -> ciflow/inductor/169432 2025-12-04T09:21:04.6173028Z * [new tag] ciflow/inductor/169436 -> ciflow/inductor/169436 2025-12-04T09:21:04.6174031Z * [new tag] ciflow/inductor/169437 -> ciflow/inductor/169437 2025-12-04T09:21:04.6175001Z * [new tag] ciflow/inductor/169438 -> ciflow/inductor/169438 2025-12-04T09:21:04.6175980Z * [new tag] ciflow/inductor/169441 -> ciflow/inductor/169441 2025-12-04T09:21:04.6176955Z * [new tag] ciflow/inductor/169446 -> ciflow/inductor/169446 2025-12-04T09:21:04.6188403Z * [new tag] ciflow/inductor/169447 -> ciflow/inductor/169447 2025-12-04T09:21:04.6189361Z * [new tag] ciflow/inductor/169452 -> ciflow/inductor/169452 2025-12-04T09:21:04.6190333Z * [new tag] ciflow/inductor/169455 -> ciflow/inductor/169455 2025-12-04T09:21:04.6191321Z * [new tag] ciflow/inductor/169459 -> ciflow/inductor/169459 2025-12-04T09:21:04.6192347Z * [new tag] ciflow/inductor/169463 -> ciflow/inductor/169463 2025-12-04T09:21:04.6193334Z * [new tag] ciflow/inductor/169476 -> ciflow/inductor/169476 2025-12-04T09:21:04.6194394Z * [new tag] ciflow/inductor/169485 -> ciflow/inductor/169485 2025-12-04T09:21:04.6195436Z * [new tag] ciflow/inductor/169493 -> ciflow/inductor/169493 2025-12-04T09:21:04.6196486Z * [new tag] ciflow/inductor/169496 -> ciflow/inductor/169496 2025-12-04T09:21:04.6197563Z * [new tag] ciflow/inductor/169497 -> ciflow/inductor/169497 2025-12-04T09:21:04.6198601Z * [new tag] ciflow/inductor/169503 -> ciflow/inductor/169503 2025-12-04T09:21:04.6199752Z * [new tag] ciflow/inductor/169504 -> ciflow/inductor/169504 2025-12-04T09:21:04.6201442Z * [new tag] ciflow/inductor/169505 -> ciflow/inductor/169505 2025-12-04T09:21:04.6202533Z * [new tag] ciflow/inductor/169508 -> ciflow/inductor/169508 2025-12-04T09:21:04.6203650Z * [new tag] ciflow/inductor/169509 -> ciflow/inductor/169509 2025-12-04T09:21:04.6204725Z * [new tag] ciflow/inductor/169513 -> ciflow/inductor/169513 2025-12-04T09:21:04.6205794Z * [new tag] ciflow/inductor/169514 -> ciflow/inductor/169514 2025-12-04T09:21:04.6206834Z * [new tag] ciflow/inductor/169515 -> ciflow/inductor/169515 2025-12-04T09:21:04.6207899Z * [new tag] ciflow/inductor/169517 -> ciflow/inductor/169517 2025-12-04T09:21:04.6208840Z * [new tag] ciflow/inductor/169519 -> ciflow/inductor/169519 2025-12-04T09:21:04.6209814Z * [new tag] ciflow/inductor/169520 -> ciflow/inductor/169520 2025-12-04T09:21:04.6210817Z * [new tag] ciflow/inductor/169521 -> ciflow/inductor/169521 2025-12-04T09:21:04.6211824Z * [new tag] ciflow/inductor/169524 -> ciflow/inductor/169524 2025-12-04T09:21:04.6217257Z * [new tag] ciflow/inductor/169527 -> ciflow/inductor/169527 2025-12-04T09:21:04.6218521Z * [new tag] ciflow/inductor/169528 -> ciflow/inductor/169528 2025-12-04T09:21:04.6219582Z * [new tag] ciflow/inductor/169532 -> ciflow/inductor/169532 2025-12-04T09:21:04.6220595Z * [new tag] ciflow/inductor/169535 -> ciflow/inductor/169535 2025-12-04T09:21:04.6221582Z * [new tag] ciflow/inductor/169536 -> ciflow/inductor/169536 2025-12-04T09:21:04.6222805Z * [new tag] ciflow/inductor/169547 -> ciflow/inductor/169547 2025-12-04T09:21:04.6223845Z * [new tag] ciflow/inductor/169548 -> ciflow/inductor/169548 2025-12-04T09:21:04.6224933Z * [new tag] ciflow/inductor/169549 -> ciflow/inductor/169549 2025-12-04T09:21:04.6225888Z * [new tag] ciflow/inductor/169551 -> ciflow/inductor/169551 2025-12-04T09:21:04.6227156Z * [new tag] ciflow/inductor/169552 -> ciflow/inductor/169552 2025-12-04T09:21:04.6228182Z * [new tag] ciflow/inductor/169553 -> ciflow/inductor/169553 2025-12-04T09:21:04.6229209Z * [new tag] ciflow/inductor/169557 -> ciflow/inductor/169557 2025-12-04T09:21:04.6230253Z * [new tag] ciflow/inductor/3b9a386 -> ciflow/inductor/3b9a386 2025-12-04T09:21:04.6231346Z * [new tag] ciflow/inductor/3d4b92b -> ciflow/inductor/3d4b92b 2025-12-04T09:21:04.6232424Z * [new tag] ciflow/inductor/d224ac7 -> ciflow/inductor/d224ac7 2025-12-04T09:21:04.6233490Z * [new tag] ciflow/linux-aarch64/157994 -> ciflow/linux-aarch64/157994 2025-12-04T09:21:04.6234621Z * [new tag] ciflow/linux-aarch64/166075 -> ciflow/linux-aarch64/166075 2025-12-04T09:21:04.6235726Z * [new tag] ciflow/linux-aarch64/166876 -> ciflow/linux-aarch64/166876 2025-12-04T09:21:04.6236768Z * [new tag] ciflow/linux-aarch64/167981 -> ciflow/linux-aarch64/167981 2025-12-04T09:21:04.6237823Z * [new tag] ciflow/mps/166254 -> ciflow/mps/166254 2025-12-04T09:21:04.6238853Z * [new tag] ciflow/mps/169017 -> ciflow/mps/169017 2025-12-04T09:21:04.6239752Z * [new tag] ciflow/mps/169372 -> ciflow/mps/169372 2025-12-04T09:21:04.6240665Z * [new tag] ciflow/mps/169478 -> ciflow/mps/169478 2025-12-04T09:21:04.6241653Z * [new tag] ciflow/op-benchmark/157994 -> ciflow/op-benchmark/157994 2025-12-04T09:21:04.6242736Z * [new tag] ciflow/op-benchmark/166075 -> ciflow/op-benchmark/166075 2025-12-04T09:21:04.6243805Z * [new tag] ciflow/op-benchmark/169544 -> ciflow/op-benchmark/169544 2025-12-04T09:21:04.6244986Z * [new tag] ciflow/periodic-rocm-mi200/165997 -> ciflow/periodic-rocm-mi200/165997 2025-12-04T09:21:04.6246282Z * [new tag] ciflow/periodic-rocm-mi200/166517 -> ciflow/periodic-rocm-mi200/166517 2025-12-04T09:21:04.6247566Z * [new tag] ciflow/periodic-rocm-mi200/169063 -> ciflow/periodic-rocm-mi200/169063 2025-12-04T09:21:04.6248881Z * [new tag] ciflow/periodic-rocm-mi200/169425 -> ciflow/periodic-rocm-mi200/169425 2025-12-04T09:21:04.6250291Z * [new tag] ciflow/periodic-rocm-mi300/166517 -> ciflow/periodic-rocm-mi300/166517 2025-12-04T09:21:04.6251637Z * [new tag] ciflow/periodic-rocm-mi300/169063 -> ciflow/periodic-rocm-mi300/169063 2025-12-04T09:21:04.6253112Z * [new tag] ciflow/periodic-rocm-mi300/169425 -> ciflow/periodic-rocm-mi300/169425 2025-12-04T09:21:04.6254408Z * [new tag] ciflow/periodic/054a2fd -> ciflow/periodic/054a2fd 2025-12-04T09:21:04.6255509Z * [new tag] ciflow/periodic/167207 -> ciflow/periodic/167207 2025-12-04T09:21:04.6256549Z * [new tag] ciflow/periodic/167978 -> ciflow/periodic/167978 2025-12-04T09:21:04.6257598Z * [new tag] ciflow/periodic/168096 -> ciflow/periodic/168096 2025-12-04T09:21:04.6258824Z * [new tag] ciflow/periodic/169286 -> ciflow/periodic/169286 2025-12-04T09:21:04.6259902Z * [new tag] ciflow/periodic/2a6d37d -> ciflow/periodic/2a6d37d 2025-12-04T09:21:04.6260962Z * [new tag] ciflow/periodic/317eeb8 -> ciflow/periodic/317eeb8 2025-12-04T09:21:04.6262145Z * [new tag] ciflow/periodic/3c32 -> ciflow/periodic/3c32 2025-12-04T09:21:04.6263202Z * [new tag] ciflow/periodic/3e98831 -> ciflow/periodic/3e98831 2025-12-04T09:21:04.6264988Z * [new tag] ciflow/periodic/7c648509a7470ace9fb2bae960dd4790f7e943e9 -> ciflow/periodic/7c648509a7470ace9fb2bae960dd4790f7e943e9 2025-12-04T09:21:04.6267118Z * [new tag] ciflow/periodic/94512-point -> ciflow/periodic/94512-point 2025-12-04T09:21:04.6268406Z * [new tag] ciflow/periodic/csl/test87519 -> ciflow/periodic/csl/test87519 2025-12-04T09:21:04.6269754Z * [new tag] ciflow/periodic/csltest88275 -> ciflow/periodic/csltest88275 2025-12-04T09:21:04.6271035Z * [new tag] ciflow/periodic/csltest88761 -> ciflow/periodic/csltest88761 2025-12-04T09:21:04.6272352Z * [new tag] ciflow/periodic/release_1.12 -> ciflow/periodic/release_1.12 2025-12-04T09:21:04.6273712Z * [new tag] ciflow/periodic/release_1.12.0 -> ciflow/periodic/release_1.12.0 2025-12-04T09:21:04.6280012Z * [new tag] ciflow/periodic/sha-ec5b83 -> ciflow/periodic/sha-ec5b83 2025-12-04T09:21:04.6281046Z * [new tag] ciflow/pull/167207 -> ciflow/pull/167207 2025-12-04T09:21:04.6282260Z * [new tag] ciflow/quantization-periodic/169207 -> ciflow/quantization-periodic/169207 2025-12-04T09:21:04.6283558Z * [new tag] ciflow/rocm-mi200/165545 -> ciflow/rocm-mi200/165545 2025-12-04T09:21:04.6284649Z * [new tag] ciflow/rocm-mi200/165997 -> ciflow/rocm-mi200/165997 2025-12-04T09:21:04.6285725Z * [new tag] ciflow/rocm-mi200/168096 -> ciflow/rocm-mi200/168096 2025-12-04T09:21:04.6286762Z * [new tag] ciflow/rocm-mi200/168275 -> ciflow/rocm-mi200/168275 2025-12-04T09:21:04.6287905Z * [new tag] ciflow/rocm-mi200/169063 -> ciflow/rocm-mi200/169063 2025-12-04T09:21:04.6288935Z * [new tag] ciflow/rocm-mi200/169356 -> ciflow/rocm-mi200/169356 2025-12-04T09:21:04.6289965Z * [new tag] ciflow/rocm-mi200/169425 -> ciflow/rocm-mi200/169425 2025-12-04T09:21:04.6291036Z * [new tag] ciflow/rocm-mi300/165545 -> ciflow/rocm-mi300/165545 2025-12-04T09:21:04.6292076Z * [new tag] ciflow/rocm-mi300/167157 -> ciflow/rocm-mi300/167157 2025-12-04T09:21:04.6293196Z * [new tag] ciflow/rocm-mi300/168096 -> ciflow/rocm-mi300/168096 2025-12-04T09:21:04.6294233Z * [new tag] ciflow/rocm-mi300/169063 -> ciflow/rocm-mi300/169063 2025-12-04T09:21:04.6295289Z * [new tag] ciflow/rocm-mi300/169425 -> ciflow/rocm-mi300/169425 2025-12-04T09:21:04.6296339Z * [new tag] ciflow/rocm-mi355/167157 -> ciflow/rocm-mi355/167157 2025-12-04T09:21:04.6297331Z * [new tag] ciflow/rocm-mi355/168275 -> ciflow/rocm-mi355/168275 2025-12-04T09:21:04.6298379Z * [new tag] ciflow/rocm-mi355/169425 -> ciflow/rocm-mi355/169425 2025-12-04T09:21:04.6299469Z * [new tag] ciflow/rocm-navi31/168275 -> ciflow/rocm-navi31/168275 2025-12-04T09:21:04.6300568Z * [new tag] ciflow/rocm-navi31/169425 -> ciflow/rocm-navi31/169425 2025-12-04T09:21:04.6302307Z * [new tag] ciflow/rocm/115316 -> ciflow/rocm/115316 2025-12-04T09:21:04.6303278Z * [new tag] ciflow/rocm/148492 -> ciflow/rocm/148492 2025-12-04T09:21:04.6304201Z * [new tag] ciflow/rocm/160685 -> ciflow/rocm/160685 2025-12-04T09:21:04.6305370Z * [new tag] ciflow/rocm/161607 -> ciflow/rocm/161607 2025-12-04T09:21:04.6306320Z * [new tag] ciflow/rocm/162052 -> ciflow/rocm/162052 2025-12-04T09:21:04.6307444Z * [new tag] ciflow/rocm/165997 -> ciflow/rocm/165997 2025-12-04T09:21:04.6308546Z * [new tag] ciflow/rocm/166165 -> ciflow/rocm/166165 2025-12-04T09:21:04.6309483Z * [new tag] ciflow/rocm/166517 -> ciflow/rocm/166517 2025-12-04T09:21:04.6310469Z * [new tag] ciflow/rocm/167207 -> ciflow/rocm/167207 2025-12-04T09:21:04.6311480Z * [new tag] ciflow/rocm/167536 -> ciflow/rocm/167536 2025-12-04T09:21:04.6312504Z * [new tag] ciflow/rocm/167781 -> ciflow/rocm/167781 2025-12-04T09:21:04.6313498Z * [new tag] ciflow/rocm/167989 -> ciflow/rocm/167989 2025-12-04T09:21:04.6314509Z * [new tag] ciflow/rocm/168073 -> ciflow/rocm/168073 2025-12-04T09:21:04.6315519Z * [new tag] ciflow/rocm/168195 -> ciflow/rocm/168195 2025-12-04T09:21:04.6316534Z * [new tag] ciflow/rocm/168939 -> ciflow/rocm/168939 2025-12-04T09:21:04.6317503Z * [new tag] ciflow/rocm/168971 -> ciflow/rocm/168971 2025-12-04T09:21:04.6318584Z * [new tag] ciflow/rocm/169024 -> ciflow/rocm/169024 2025-12-04T09:21:04.6319524Z * [new tag] ciflow/rocm/169200 -> ciflow/rocm/169200 2025-12-04T09:21:04.6320443Z * [new tag] ciflow/rocm/169216 -> ciflow/rocm/169216 2025-12-04T09:21:04.6321412Z * [new tag] ciflow/rocm/169312 -> ciflow/rocm/169312 2025-12-04T09:21:04.6322329Z * [new tag] ciflow/rocm/169380 -> ciflow/rocm/169380 2025-12-04T09:21:04.6323274Z * [new tag] ciflow/rocm/169427 -> ciflow/rocm/169427 2025-12-04T09:21:04.6324225Z * [new tag] ciflow/rocm/169455 -> ciflow/rocm/169455 2025-12-04T09:21:04.6325181Z * [new tag] ciflow/rocm/169470 -> ciflow/rocm/169470 2025-12-04T09:21:04.6326139Z * [new tag] ciflow/rocm/169471 -> ciflow/rocm/169471 2025-12-04T09:21:04.6327120Z * [new tag] ciflow/rocm/169472 -> ciflow/rocm/169472 2025-12-04T09:21:04.6328063Z * [new tag] ciflow/rocm/169514 -> ciflow/rocm/169514 2025-12-04T09:21:04.6329000Z * [new tag] ciflow/slow/01c7106 -> ciflow/slow/01c7106 2025-12-04T09:21:04.6329968Z * [new tag] ciflow/slow/0577043 -> ciflow/slow/0577043 2025-12-04T09:21:04.6331701Z * [new tag] ciflow/slow/0d5b74da0cab798fbfdb9caa53fad816999c8386-sdym -> ciflow/slow/0d5b74da0cab798fbfdb9caa53fad816999c8386-sdym 2025-12-04T09:21:04.6333425Z * [new tag] ciflow/slow/0e81104 -> ciflow/slow/0e81104 2025-12-04T09:21:04.6334427Z * [new tag] ciflow/slow/167207 -> ciflow/slow/167207 2025-12-04T09:21:04.6335417Z * [new tag] ciflow/slow/168050 -> ciflow/slow/168050 2025-12-04T09:21:04.6336379Z * [new tag] ciflow/slow/1732077 -> ciflow/slow/1732077 2025-12-04T09:21:04.6337445Z * [new tag] ciflow/slow/187eb7c -> ciflow/slow/187eb7c 2025-12-04T09:21:04.6338001Z * [new tag] ciflow/slow/1faef89 -> ciflow/slow/1faef89 2025-12-04T09:21:04.6338546Z * [new tag] ciflow/slow/3920ec1 -> ciflow/slow/3920ec1 2025-12-04T09:21:04.6339077Z * [new tag] ciflow/slow/3b7c6b2 -> ciflow/slow/3b7c6b2 2025-12-04T09:21:04.6339619Z * [new tag] ciflow/slow/59a3759 -> ciflow/slow/59a3759 2025-12-04T09:21:04.6340165Z * [new tag] ciflow/slow/70ef0bb -> ciflow/slow/70ef0bb 2025-12-04T09:21:04.6341107Z * [new tag] ciflow/slow/788ff06 -> ciflow/slow/788ff06 2025-12-04T09:21:04.6342743Z * [new tag] ciflow/slow/8751002215790a3a88750faa8f4366933e296693-sdym -> ciflow/slow/8751002215790a3a88750faa8f4366933e296693-sdym 2025-12-04T09:21:04.6344368Z * [new tag] ciflow/slow/9d85864 -> ciflow/slow/9d85864 2025-12-04T09:21:04.6345486Z * [new tag] ciflow/slow/9ffad5b -> ciflow/slow/9ffad5b 2025-12-04T09:21:04.6346470Z * [new tag] ciflow/slow/a206e8b -> ciflow/slow/a206e8b 2025-12-04T09:21:04.6347539Z * [new tag] ciflow/slow/a837609 -> ciflow/slow/a837609 2025-12-04T09:21:04.6348514Z * [new tag] ciflow/slow/af841f3 -> ciflow/slow/af841f3 2025-12-04T09:21:04.6350291Z * [new tag] ciflow/slow/da3aba1e46157c4df504b067477cdf2b3c96b194-sdym -> ciflow/slow/da3aba1e46157c4df504b067477cdf2b3c96b194-sdym 2025-12-04T09:21:04.6352164Z * [new tag] ciflow/torchbench/168175 -> ciflow/torchbench/168175 2025-12-04T09:21:04.6353236Z * [new tag] ciflow/trunk/148492 -> ciflow/trunk/148492 2025-12-04T09:21:04.6354235Z * [new tag] ciflow/trunk/157149 -> ciflow/trunk/157149 2025-12-04T09:21:04.6355247Z * [new tag] ciflow/trunk/157994 -> ciflow/trunk/157994 2025-12-04T09:21:04.6356216Z * [new tag] ciflow/trunk/159718 -> ciflow/trunk/159718 2025-12-04T09:21:04.6357201Z * [new tag] ciflow/trunk/160685 -> ciflow/trunk/160685 2025-12-04T09:21:04.6358170Z * [new tag] ciflow/trunk/160729 -> ciflow/trunk/160729 2025-12-04T09:21:04.6359166Z * [new tag] ciflow/trunk/162275 -> ciflow/trunk/162275 2025-12-04T09:21:04.6360037Z * [new tag] ciflow/trunk/162795 -> ciflow/trunk/162795 2025-12-04T09:21:04.6360946Z * [new tag] ciflow/trunk/163245 -> ciflow/trunk/163245 2025-12-04T09:21:04.6361864Z * [new tag] ciflow/trunk/163942 -> ciflow/trunk/163942 2025-12-04T09:21:04.6362799Z * [new tag] ciflow/trunk/165274 -> ciflow/trunk/165274 2025-12-04T09:21:04.6363741Z * [new tag] ciflow/trunk/165483 -> ciflow/trunk/165483 2025-12-04T09:21:04.6364719Z * [new tag] ciflow/trunk/165728 -> ciflow/trunk/165728 2025-12-04T09:21:04.6365655Z * [new tag] ciflow/trunk/165922 -> ciflow/trunk/165922 2025-12-04T09:21:04.6366586Z * [new tag] ciflow/trunk/166075 -> ciflow/trunk/166075 2025-12-04T09:21:04.6367505Z * [new tag] ciflow/trunk/166165 -> ciflow/trunk/166165 2025-12-04T09:21:04.6368371Z * [new tag] ciflow/trunk/166829 -> ciflow/trunk/166829 2025-12-04T09:21:04.6369265Z * [new tag] ciflow/trunk/166843 -> ciflow/trunk/166843 2025-12-04T09:21:04.6370178Z * [new tag] ciflow/trunk/166876 -> ciflow/trunk/166876 2025-12-04T09:21:04.6371053Z * [new tag] ciflow/trunk/167207 -> ciflow/trunk/167207 2025-12-04T09:21:04.6371940Z * [new tag] ciflow/trunk/167536 -> ciflow/trunk/167536 2025-12-04T09:21:04.6372862Z * [new tag] ciflow/trunk/167552 -> ciflow/trunk/167552 2025-12-04T09:21:04.6373804Z * [new tag] ciflow/trunk/167555 -> ciflow/trunk/167555 2025-12-04T09:21:04.6374730Z * [new tag] ciflow/trunk/167599 -> ciflow/trunk/167599 2025-12-04T09:21:04.6375586Z * [new tag] ciflow/trunk/167659 -> ciflow/trunk/167659 2025-12-04T09:21:04.6376466Z * [new tag] ciflow/trunk/167672 -> ciflow/trunk/167672 2025-12-04T09:21:04.6377430Z * [new tag] ciflow/trunk/167742 -> ciflow/trunk/167742 2025-12-04T09:21:04.6378532Z * [new tag] ciflow/trunk/167781 -> ciflow/trunk/167781 2025-12-04T09:21:04.6379449Z * [new tag] ciflow/trunk/167837 -> ciflow/trunk/167837 2025-12-04T09:21:04.6380377Z * [new tag] ciflow/trunk/167887 -> ciflow/trunk/167887 2025-12-04T09:21:04.6381320Z * [new tag] ciflow/trunk/167978 -> ciflow/trunk/167978 2025-12-04T09:21:04.6382392Z * [new tag] ciflow/trunk/168050 -> ciflow/trunk/168050 2025-12-04T09:21:04.6383335Z * [new tag] ciflow/trunk/168051 -> ciflow/trunk/168051 2025-12-04T09:21:04.6384199Z * [new tag] ciflow/trunk/168096 -> ciflow/trunk/168096 2025-12-04T09:21:04.6385154Z * [new tag] ciflow/trunk/168127 -> ciflow/trunk/168127 2025-12-04T09:21:04.6386121Z * [new tag] ciflow/trunk/168157 -> ciflow/trunk/168157 2025-12-04T09:21:04.6387374Z * [new tag] ciflow/trunk/168175 -> ciflow/trunk/168175 2025-12-04T09:21:04.6388374Z * [new tag] ciflow/trunk/168209 -> ciflow/trunk/168209 2025-12-04T09:21:04.6389383Z * [new tag] ciflow/trunk/168213 -> ciflow/trunk/168213 2025-12-04T09:21:04.6390373Z * [new tag] ciflow/trunk/168226 -> ciflow/trunk/168226 2025-12-04T09:21:04.6391363Z * [new tag] ciflow/trunk/168262 -> ciflow/trunk/168262 2025-12-04T09:21:04.6392378Z * [new tag] ciflow/trunk/168275 -> ciflow/trunk/168275 2025-12-04T09:21:04.6393384Z * [new tag] ciflow/trunk/168328 -> ciflow/trunk/168328 2025-12-04T09:21:04.6394381Z * [new tag] ciflow/trunk/168368 -> ciflow/trunk/168368 2025-12-04T09:21:04.6395352Z * [new tag] ciflow/trunk/168917 -> ciflow/trunk/168917 2025-12-04T09:21:04.6396351Z * [new tag] ciflow/trunk/168933 -> ciflow/trunk/168933 2025-12-04T09:21:04.6397365Z * [new tag] ciflow/trunk/168941 -> ciflow/trunk/168941 2025-12-04T09:21:04.6398353Z * [new tag] ciflow/trunk/168955 -> ciflow/trunk/168955 2025-12-04T09:21:04.6399240Z * [new tag] ciflow/trunk/168980 -> ciflow/trunk/168980 2025-12-04T09:21:04.6399950Z * [new tag] ciflow/trunk/169004 -> ciflow/trunk/169004 2025-12-04T09:21:04.6400499Z * [new tag] ciflow/trunk/169006 -> ciflow/trunk/169006 2025-12-04T09:21:04.6401407Z * [new tag] ciflow/trunk/169023 -> ciflow/trunk/169023 2025-12-04T09:21:04.6401953Z * [new tag] ciflow/trunk/169025 -> ciflow/trunk/169025 2025-12-04T09:21:04.6402494Z * [new tag] ciflow/trunk/169048 -> ciflow/trunk/169048 2025-12-04T09:21:04.6403031Z * [new tag] ciflow/trunk/169066 -> ciflow/trunk/169066 2025-12-04T09:21:04.6403558Z * [new tag] ciflow/trunk/169091 -> ciflow/trunk/169091 2025-12-04T09:21:04.6404313Z * [new tag] ciflow/trunk/169102 -> ciflow/trunk/169102 2025-12-04T09:21:04.6405281Z * [new tag] ciflow/trunk/169103 -> ciflow/trunk/169103 2025-12-04T09:21:04.6406228Z * [new tag] ciflow/trunk/169125 -> ciflow/trunk/169125 2025-12-04T09:21:04.6407164Z * [new tag] ciflow/trunk/169139 -> ciflow/trunk/169139 2025-12-04T09:21:04.6408103Z * [new tag] ciflow/trunk/169148 -> ciflow/trunk/169148 2025-12-04T09:21:04.6409067Z * [new tag] ciflow/trunk/169151 -> ciflow/trunk/169151 2025-12-04T09:21:04.6409995Z * [new tag] ciflow/trunk/169156 -> ciflow/trunk/169156 2025-12-04T09:21:04.6410966Z * [new tag] ciflow/trunk/169176 -> ciflow/trunk/169176 2025-12-04T09:21:04.6411958Z * [new tag] ciflow/trunk/169204 -> ciflow/trunk/169204 2025-12-04T09:21:04.6413254Z * [new tag] ciflow/trunk/169207 -> ciflow/trunk/169207 2025-12-04T09:21:04.6414211Z * [new tag] ciflow/trunk/169211 -> ciflow/trunk/169211 2025-12-04T09:21:04.6415164Z * [new tag] ciflow/trunk/169231 -> ciflow/trunk/169231 2025-12-04T09:21:04.6416300Z * [new tag] ciflow/trunk/169260 -> ciflow/trunk/169260 2025-12-04T09:21:04.6417297Z * [new tag] ciflow/trunk/169271 -> ciflow/trunk/169271 2025-12-04T09:21:04.6418254Z * [new tag] ciflow/trunk/169280 -> ciflow/trunk/169280 2025-12-04T09:21:04.6419116Z * [new tag] ciflow/trunk/169281 -> ciflow/trunk/169281 2025-12-04T09:21:04.6420045Z * [new tag] ciflow/trunk/169286 -> ciflow/trunk/169286 2025-12-04T09:21:04.6420935Z * [new tag] ciflow/trunk/169293 -> ciflow/trunk/169293 2025-12-04T09:21:04.6421876Z * [new tag] ciflow/trunk/169296 -> ciflow/trunk/169296 2025-12-04T09:21:04.6422827Z * [new tag] ciflow/trunk/169304 -> ciflow/trunk/169304 2025-12-04T09:21:04.6423777Z * [new tag] ciflow/trunk/169305 -> ciflow/trunk/169305 2025-12-04T09:21:04.6424737Z * [new tag] ciflow/trunk/169312 -> ciflow/trunk/169312 2025-12-04T09:21:04.6425690Z * [new tag] ciflow/trunk/169328 -> ciflow/trunk/169328 2025-12-04T09:21:04.6426775Z * [new tag] ciflow/trunk/169343 -> ciflow/trunk/169343 2025-12-04T09:21:04.6427932Z * [new tag] ciflow/trunk/169355 -> ciflow/trunk/169355 2025-12-04T09:21:04.6428905Z * [new tag] ciflow/trunk/169370 -> ciflow/trunk/169370 2025-12-04T09:21:04.6429888Z * [new tag] ciflow/trunk/169379 -> ciflow/trunk/169379 2025-12-04T09:21:04.6430873Z * [new tag] ciflow/trunk/169380 -> ciflow/trunk/169380 2025-12-04T09:21:04.6431879Z * [new tag] ciflow/trunk/169385 -> ciflow/trunk/169385 2025-12-04T09:21:04.6432848Z * [new tag] ciflow/trunk/169387 -> ciflow/trunk/169387 2025-12-04T09:21:04.6433813Z * [new tag] ciflow/trunk/169410 -> ciflow/trunk/169410 2025-12-04T09:21:04.6434705Z * [new tag] ciflow/trunk/169412 -> ciflow/trunk/169412 2025-12-04T09:21:04.6435606Z * [new tag] ciflow/trunk/169418 -> ciflow/trunk/169418 2025-12-04T09:21:04.6436567Z * [new tag] ciflow/trunk/169423 -> ciflow/trunk/169423 2025-12-04T09:21:04.6437484Z * [new tag] ciflow/trunk/169427 -> ciflow/trunk/169427 2025-12-04T09:21:04.6438430Z * [new tag] ciflow/trunk/169430 -> ciflow/trunk/169430 2025-12-04T09:21:04.6439467Z * [new tag] ciflow/trunk/169437 -> ciflow/trunk/169437 2025-12-04T09:21:04.6440357Z * [new tag] ciflow/trunk/169442 -> ciflow/trunk/169442 2025-12-04T09:21:04.6441272Z * [new tag] ciflow/trunk/169452 -> ciflow/trunk/169452 2025-12-04T09:21:04.6442217Z * [new tag] ciflow/trunk/169454 -> ciflow/trunk/169454 2025-12-04T09:21:04.6443188Z * [new tag] ciflow/trunk/169459 -> ciflow/trunk/169459 2025-12-04T09:21:04.6444139Z * [new tag] ciflow/trunk/169474 -> ciflow/trunk/169474 2025-12-04T09:21:04.6445103Z * [new tag] ciflow/trunk/169475 -> ciflow/trunk/169475 2025-12-04T09:21:04.6446106Z * [new tag] ciflow/trunk/169476 -> ciflow/trunk/169476 2025-12-04T09:21:04.6447050Z * [new tag] ciflow/trunk/169487 -> ciflow/trunk/169487 2025-12-04T09:21:04.6447981Z * [new tag] ciflow/trunk/169497 -> ciflow/trunk/169497 2025-12-04T09:21:04.6449109Z * [new tag] ciflow/trunk/169503 -> ciflow/trunk/169503 2025-12-04T09:21:04.6450037Z * [new tag] ciflow/trunk/169505 -> ciflow/trunk/169505 2025-12-04T09:21:04.6450892Z * [new tag] ciflow/trunk/169507 -> ciflow/trunk/169507 2025-12-04T09:21:04.6451779Z * [new tag] ciflow/trunk/169514 -> ciflow/trunk/169514 2025-12-04T09:21:04.6452902Z * [new tag] ciflow/trunk/169517 -> ciflow/trunk/169517 2025-12-04T09:21:04.6453810Z * [new tag] ciflow/trunk/169519 -> ciflow/trunk/169519 2025-12-04T09:21:04.6454691Z * [new tag] ciflow/trunk/169528 -> ciflow/trunk/169528 2025-12-04T09:21:04.6455572Z * [new tag] ciflow/trunk/169541 -> ciflow/trunk/169541 2025-12-04T09:21:04.6456491Z * [new tag] ciflow/trunk/169555 -> ciflow/trunk/169555 2025-12-04T09:21:04.6457354Z * [new tag] ciflow/unstable/123 -> ciflow/unstable/123 2025-12-04T09:21:04.6458246Z * [new tag] ciflow/vllm/165270 -> ciflow/vllm/165270 2025-12-04T09:21:04.6459097Z * [new tag] ciflow/vllm/165274 -> ciflow/vllm/165274 2025-12-04T09:21:04.6459987Z * [new tag] ciflow/vllm/166494 -> ciflow/vllm/166494 2025-12-04T09:21:04.6460854Z * [new tag] ciflow/vllm/169219 -> ciflow/vllm/169219 2025-12-04T09:21:04.6461745Z * [new tag] ciflow/vllm/169220 -> ciflow/vllm/169220 2025-12-04T09:21:04.6467798Z * [new tag] ciflow/xpu/157994 -> ciflow/xpu/157994 2025-12-04T09:21:04.6468340Z * [new tag] ciflow/xpu/159718 -> ciflow/xpu/159718 2025-12-04T09:21:04.6468862Z * [new tag] ciflow/xpu/161940 -> ciflow/xpu/161940 2025-12-04T09:21:04.6469395Z * [new tag] ciflow/xpu/163251 -> ciflow/xpu/163251 2025-12-04T09:21:04.6469935Z * [new tag] ciflow/xpu/166829 -> ciflow/xpu/166829 2025-12-04T09:21:04.6470449Z * [new tag] ciflow/xpu/166843 -> ciflow/xpu/166843 2025-12-04T09:21:04.6470979Z * [new tag] ciflow/xpu/167972 -> ciflow/xpu/167972 2025-12-04T09:21:04.6471506Z * [new tag] ciflow/xpu/167981 -> ciflow/xpu/167981 2025-12-04T09:21:04.6472042Z * [new tag] ciflow/xpu/168213 -> ciflow/xpu/168213 2025-12-04T09:21:04.6472561Z * [new tag] ciflow/xpu/168262 -> ciflow/xpu/168262 2025-12-04T09:21:04.6473092Z * [new tag] ciflow/xpu/168328 -> ciflow/xpu/168328 2025-12-04T09:21:04.6473620Z * [new tag] ciflow/xpu/168950 -> ciflow/xpu/168950 2025-12-04T09:21:04.6474145Z * [new tag] ciflow/xpu/169039 -> ciflow/xpu/169039 2025-12-04T09:21:04.6474661Z * [new tag] ciflow/xpu/169200 -> ciflow/xpu/169200 2025-12-04T09:21:04.6475193Z * [new tag] ciflow/xpu/169203 -> ciflow/xpu/169203 2025-12-04T09:21:04.6475722Z * [new tag] ciflow/xpu/169230 -> ciflow/xpu/169230 2025-12-04T09:21:04.6476234Z * [new tag] ciflow/xpu/169231 -> ciflow/xpu/169231 2025-12-04T09:21:04.6476761Z * [new tag] ciflow/xpu/169241 -> ciflow/xpu/169241 2025-12-04T09:21:04.6477289Z * [new tag] ciflow/xpu/169280 -> ciflow/xpu/169280 2025-12-04T09:21:04.6477812Z * [new tag] ciflow/xpu/169296 -> ciflow/xpu/169296 2025-12-04T09:21:04.6478323Z * [new tag] ciflow/xpu/169353 -> ciflow/xpu/169353 2025-12-04T09:21:04.6478953Z * [new tag] ciflow/xpu/169410 -> ciflow/xpu/169410 2025-12-04T09:21:04.6479462Z * [new tag] ciflow/xpu/169442 -> ciflow/xpu/169442 2025-12-04T09:21:04.6479973Z * [new tag] ciflow/xpu/169555 -> ciflow/xpu/169555 2025-12-04T09:21:04.6480581Z * [new tag] cslpull75 -> cslpull75 2025-12-04T09:21:04.6481057Z * [new tag] cslpull76 -> cslpull76 2025-12-04T09:21:04.6481521Z * [new tag] cslpull77 -> cslpull77 2025-12-04T09:21:04.6482037Z * [new tag] cslpull78 -> cslpull78 2025-12-04T09:21:04.6482502Z * [new tag] cslpull79 -> cslpull79 2025-12-04T09:21:04.6482959Z * [new tag] cslpull80 -> cslpull80 2025-12-04T09:21:04.6483423Z * [new tag] cslpull81 -> cslpull81 2025-12-04T09:21:04.6483871Z * [new tag] cslpull82 -> cslpull82 2025-12-04T09:21:04.6484331Z * [new tag] cslpull83 -> cslpull83 2025-12-04T09:21:04.6484798Z * [new tag] cslpull84 -> cslpull84 2025-12-04T09:21:04.6485257Z * [new tag] cslpull85 -> cslpull85 2025-12-04T09:21:04.6485714Z * [new tag] cslpull86 -> cslpull86 2025-12-04T09:21:04.6486193Z * [new tag] cslpull87 -> cslpull87 2025-12-04T09:21:04.6486654Z * [new tag] cslpull88 -> cslpull88 2025-12-04T09:21:04.6487112Z * [new tag] cslpull89 -> cslpull89 2025-12-04T09:21:04.6487571Z * [new tag] cslpull90 -> cslpull90 2025-12-04T09:21:04.6488032Z * [new tag] cslpull91 -> cslpull91 2025-12-04T09:21:04.6488493Z * [new tag] cslpull92 -> cslpull92 2025-12-04T09:21:04.6488945Z * [new tag] flight_5 -> flight_5 2025-12-04T09:21:04.6489409Z * [new tag] flight_5.1 -> flight_5.1 2025-12-04T09:21:04.6489877Z * [new tag] flight_5.2 -> flight_5.2 2025-12-04T09:21:04.6490327Z * [new tag] flight_5.3 -> flight_5.3 2025-12-04T09:21:04.6490786Z * [new tag] forpull1 -> forpull1 2025-12-04T09:21:04.6491282Z * [new tag] malfet/tag-2ef5611 -> malfet/tag-2ef5611 2025-12-04T09:21:04.6491815Z * [new tag] malfet/tag-317b1a0 -> malfet/tag-317b1a0 2025-12-04T09:21:04.6492330Z * [new tag] malfet/tag-ec6f767 -> malfet/tag-ec6f767 2025-12-04T09:21:04.6492845Z * [new tag] nightly-binary -> nightly-binary 2025-12-04T09:21:04.6493370Z * [new tag] sqzhang_flight4_plus -> sqzhang_flight4_plus 2025-12-04T09:21:04.6493886Z * [new tag] sqzhang_flight_3 -> sqzhang_flight_3 2025-12-04T09:21:04.6494670Z * [new tag] trunk/02d8bd6974cf84b721680d773dbdb1b6f40ce272 -> trunk/02d8bd6974cf84b721680d773dbdb1b6f40ce272 2025-12-04T09:21:04.6495712Z * [new tag] trunk/066997fb38ade71e00d78e9d572e380b5f02bd3e -> trunk/066997fb38ade71e00d78e9d572e380b5f02bd3e 2025-12-04T09:21:04.6496739Z * [new tag] trunk/076e7b19fa1d481ad778d06d2b49ba57d3ce8c88 -> trunk/076e7b19fa1d481ad778d06d2b49ba57d3ce8c88 2025-12-04T09:21:04.6497758Z * [new tag] trunk/07dcc0b83db3211653a38565a24e15acdba75654 -> trunk/07dcc0b83db3211653a38565a24e15acdba75654 2025-12-04T09:21:04.6498793Z * [new tag] trunk/082e96b68dfcd16cab7cfafc4d3d055767dab3eb -> trunk/082e96b68dfcd16cab7cfafc4d3d055767dab3eb 2025-12-04T09:21:04.6499823Z * [new tag] trunk/088048f2fea28ff7d450f65c72419ca45780d30b -> trunk/088048f2fea28ff7d450f65c72419ca45780d30b 2025-12-04T09:21:04.6501479Z * [new tag] trunk/09076941a95c76f4d9ad189d064dfd8baa39e672 -> trunk/09076941a95c76f4d9ad189d064dfd8baa39e672 2025-12-04T09:21:04.6502648Z * [new tag] trunk/0b80a4c62b94402844bf221791c096b0035c6d75 -> trunk/0b80a4c62b94402844bf221791c096b0035c6d75 2025-12-04T09:21:04.6503680Z * [new tag] trunk/0bbbdf1750567a980634ad907a325357ba8ba8f2 -> trunk/0bbbdf1750567a980634ad907a325357ba8ba8f2 2025-12-04T09:21:04.6504846Z * [new tag] trunk/0c281dd78773b2bc17c58ead0e4cd4ac46e775c5 -> trunk/0c281dd78773b2bc17c58ead0e4cd4ac46e775c5 2025-12-04T09:21:04.6505901Z * [new tag] trunk/135f3753c418a6879b1954904184937b67e61688 -> trunk/135f3753c418a6879b1954904184937b67e61688 2025-12-04T09:21:04.6507088Z * [new tag] trunk/15da21026cb13cd20257dc9e96830db108743c10 -> trunk/15da21026cb13cd20257dc9e96830db108743c10 2025-12-04T09:21:04.6508136Z * [new tag] trunk/166efdad2ac827f30fb02504c6017520257f88ec -> trunk/166efdad2ac827f30fb02504c6017520257f88ec 2025-12-04T09:21:04.6509181Z * [new tag] trunk/174272c15fae553d8488140af931f7d8050a313f -> trunk/174272c15fae553d8488140af931f7d8050a313f 2025-12-04T09:21:04.6510257Z * [new tag] trunk/18f3ca08f13b8de61307f5e8cd7d4cccb67e9d11 -> trunk/18f3ca08f13b8de61307f5e8cd7d4cccb67e9d11 2025-12-04T09:21:04.6511343Z * [new tag] trunk/1902eddfe655a15ebcf2c72bd81ade110fdeef63 -> trunk/1902eddfe655a15ebcf2c72bd81ade110fdeef63 2025-12-04T09:21:04.6512405Z * [new tag] trunk/195f92e98d3d66738577f11f22c4b5c8a1c76dd5 -> trunk/195f92e98d3d66738577f11f22c4b5c8a1c76dd5 2025-12-04T09:21:04.6513470Z * [new tag] trunk/1aa13e17de39e3c768ea7aebaad166ce72a06676 -> trunk/1aa13e17de39e3c768ea7aebaad166ce72a06676 2025-12-04T09:21:04.6514549Z * [new tag] trunk/1afe2832f58e24e54a5bfda5a5afa9b96fdea40e -> trunk/1afe2832f58e24e54a5bfda5a5afa9b96fdea40e 2025-12-04T09:21:04.6515624Z * [new tag] trunk/1c87554d74140eaee964ca8b1832cede67f5f520 -> trunk/1c87554d74140eaee964ca8b1832cede67f5f520 2025-12-04T09:21:04.6516678Z * [new tag] trunk/1ccb743b7b5be955f49736c162c4f5004b8a0dd8 -> trunk/1ccb743b7b5be955f49736c162c4f5004b8a0dd8 2025-12-04T09:21:04.6517732Z * [new tag] trunk/1cee47d6ce0a02227185b566593f002dd639ca0c -> trunk/1cee47d6ce0a02227185b566593f002dd639ca0c 2025-12-04T09:21:04.6518890Z * [new tag] trunk/1d21b4df2babe322e5d085ceb6de884eb260a62d -> trunk/1d21b4df2babe322e5d085ceb6de884eb260a62d 2025-12-04T09:21:04.6519932Z * [new tag] trunk/1e34fb2550e4aa650314f7a6d9f6daf4da7478a8 -> trunk/1e34fb2550e4aa650314f7a6d9f6daf4da7478a8 2025-12-04T09:21:04.6520969Z * [new tag] trunk/1e526fb5b1d93bfc70691c5c3955fdffc1b7b7de -> trunk/1e526fb5b1d93bfc70691c5c3955fdffc1b7b7de 2025-12-04T09:21:04.6521990Z * [new tag] trunk/1ee32a8b1f554a312d79bad01ded24f38cd95543 -> trunk/1ee32a8b1f554a312d79bad01ded24f38cd95543 2025-12-04T09:21:04.6523015Z * [new tag] trunk/201e2c4117eb9744594dad6a5c18213d7b4705d7 -> trunk/201e2c4117eb9744594dad6a5c18213d7b4705d7 2025-12-04T09:21:04.6524118Z * [new tag] trunk/2353a0f60eb4b4cb6675907a7fa9fbedc1c02e7f -> trunk/2353a0f60eb4b4cb6675907a7fa9fbedc1c02e7f 2025-12-04T09:21:04.6529808Z * [new tag] trunk/285779b1621cf9f073a062b0889a642d200308d9 -> trunk/285779b1621cf9f073a062b0889a642d200308d9 2025-12-04T09:21:04.6530848Z * [new tag] trunk/2887faaec6295d081580d09fce161201826c6d87 -> trunk/2887faaec6295d081580d09fce161201826c6d87 2025-12-04T09:21:04.6531894Z * [new tag] trunk/296e67c92635443c67b11c0ae1bd045f03ebb7bc -> trunk/296e67c92635443c67b11c0ae1bd045f03ebb7bc 2025-12-04T09:21:04.6532956Z * [new tag] trunk/29856679769b3dede478767e2fe6cfb51197cb25 -> trunk/29856679769b3dede478767e2fe6cfb51197cb25 2025-12-04T09:21:04.6534004Z * [new tag] trunk/29e5455a4740c326ab187c7aa7b5ef98034ea563 -> trunk/29e5455a4740c326ab187c7aa7b5ef98034ea563 2025-12-04T09:21:04.6535047Z * [new tag] trunk/2ac3ef882afb23136adc188975f0a8802fc68adf -> trunk/2ac3ef882afb23136adc188975f0a8802fc68adf 2025-12-04T09:21:04.6536242Z * [new tag] trunk/2bec68e73b64715354af076ad309335f943e36cd -> trunk/2bec68e73b64715354af076ad309335f943e36cd 2025-12-04T09:21:04.6537296Z * [new tag] trunk/2c87367e6f88662cd5cedbd1537748b7948c38e1 -> trunk/2c87367e6f88662cd5cedbd1537748b7948c38e1 2025-12-04T09:21:04.6538417Z * [new tag] trunk/2d1f78fe3ec13820f136a2e0336da12a25f41708 -> trunk/2d1f78fe3ec13820f136a2e0336da12a25f41708 2025-12-04T09:21:04.6539475Z * [new tag] trunk/2df6058f116a65722a0e03073402feb242572d35 -> trunk/2df6058f116a65722a0e03073402feb242572d35 2025-12-04T09:21:04.6540618Z * [new tag] trunk/2e0c2e170fe658c440775c8e5c44228aafcc47ec -> trunk/2e0c2e170fe658c440775c8e5c44228aafcc47ec 2025-12-04T09:21:04.6541657Z * [new tag] trunk/2f9b7dad7b5419b063bd0f2e204de192720ebb94 -> trunk/2f9b7dad7b5419b063bd0f2e204de192720ebb94 2025-12-04T09:21:04.6542689Z * [new tag] trunk/305168768a95d69c444df5cd334bb774edfe06f1 -> trunk/305168768a95d69c444df5cd334bb774edfe06f1 2025-12-04T09:21:04.6543699Z * [new tag] trunk/31fc12773026e8e00f054dd79ad9b2491e693b48 -> trunk/31fc12773026e8e00f054dd79ad9b2491e693b48 2025-12-04T09:21:04.6544713Z * [new tag] trunk/320de0c6b0a3e7c6d2693ea5c28d5d0156ba7991 -> trunk/320de0c6b0a3e7c6d2693ea5c28d5d0156ba7991 2025-12-04T09:21:04.6545740Z * [new tag] trunk/3418bd29475dff06695045fcdf93e7d0dac67da8 -> trunk/3418bd29475dff06695045fcdf93e7d0dac67da8 2025-12-04T09:21:04.6546876Z * [new tag] trunk/34a98608afa0cb5b48f0d6d30432fdd0a2614ddf -> trunk/34a98608afa0cb5b48f0d6d30432fdd0a2614ddf 2025-12-04T09:21:04.6548114Z * [new tag] trunk/35b7a9a26c5923d98aebaa41a031dae21788a9ee -> trunk/35b7a9a26c5923d98aebaa41a031dae21788a9ee 2025-12-04T09:21:04.6549160Z * [new tag] trunk/39d07dbf03a911bdd45d1af78d8638dc92074938 -> trunk/39d07dbf03a911bdd45d1af78d8638dc92074938 2025-12-04T09:21:04.6550228Z * [new tag] trunk/3cd98b4205ada151042cc7ff097a82d4a4b18725 -> trunk/3cd98b4205ada151042cc7ff097a82d4a4b18725 2025-12-04T09:21:04.6551298Z * [new tag] trunk/3d35fd20a78ff4d016fa80f4e5fad37191d7bcae -> trunk/3d35fd20a78ff4d016fa80f4e5fad37191d7bcae 2025-12-04T09:21:04.6552371Z * [new tag] trunk/409a5fee945c46a3edaf5df162812f201bfd7b2f -> trunk/409a5fee945c46a3edaf5df162812f201bfd7b2f 2025-12-04T09:21:04.6553429Z * [new tag] trunk/42e9005cda22da3f1c559c3649218cebd671027c -> trunk/42e9005cda22da3f1c559c3649218cebd671027c 2025-12-04T09:21:04.6554488Z * [new tag] trunk/43b94713bbf340d3c124fde02d0f73add4021247 -> trunk/43b94713bbf340d3c124fde02d0f73add4021247 2025-12-04T09:21:04.6555563Z * [new tag] trunk/44ac69388a4a5eb463dbd2a13f00d1e3b924566c -> trunk/44ac69388a4a5eb463dbd2a13f00d1e3b924566c 2025-12-04T09:21:04.6556619Z * [new tag] trunk/45d14e2497292be06ad36eaa1aaaf7c630a2586a -> trunk/45d14e2497292be06ad36eaa1aaaf7c630a2586a 2025-12-04T09:21:04.6557661Z * [new tag] trunk/45d310ad84854dff730c0b12e577d7998d978686 -> trunk/45d310ad84854dff730c0b12e577d7998d978686 2025-12-04T09:21:04.6558716Z * [new tag] trunk/47b28ddf7bd74b50fa93b307a7d3b183a6d77f54 -> trunk/47b28ddf7bd74b50fa93b307a7d3b183a6d77f54 2025-12-04T09:21:04.6559868Z * [new tag] trunk/481e5ab336275bd3acd5fa8a611b05b4469012af -> trunk/481e5ab336275bd3acd5fa8a611b05b4469012af 2025-12-04T09:21:04.6560893Z * [new tag] trunk/491731647f6b8a9345dcfb3bc9416aea254a7d96 -> trunk/491731647f6b8a9345dcfb3bc9416aea254a7d96 2025-12-04T09:21:04.6561905Z * [new tag] trunk/49a04d26088acc17d948ddd66920f3e16371e873 -> trunk/49a04d26088acc17d948ddd66920f3e16371e873 2025-12-04T09:21:04.6562919Z * [new tag] trunk/4bebc827c47d2f1f0fa1a417a5201a97aef3d985 -> trunk/4bebc827c47d2f1f0fa1a417a5201a97aef3d985 2025-12-04T09:21:04.6564017Z * [new tag] trunk/4c246677784c6a14bc2dbb9ff8773ef0a3a3222f -> trunk/4c246677784c6a14bc2dbb9ff8773ef0a3a3222f 2025-12-04T09:21:04.6565043Z * [new tag] trunk/4cfb47ff548b6d996641058cf04a70e311a4c3aa -> trunk/4cfb47ff548b6d996641058cf04a70e311a4c3aa 2025-12-04T09:21:04.6566126Z * [new tag] trunk/4e0061c1aa52f606dda8cfab0bd7591e588faf2c -> trunk/4e0061c1aa52f606dda8cfab0bd7591e588faf2c 2025-12-04T09:21:04.6567156Z * [new tag] trunk/4fefb8e7e942386ffac764a41b232241f82bea3a -> trunk/4fefb8e7e942386ffac764a41b232241f82bea3a 2025-12-04T09:21:04.6568174Z * [new tag] trunk/503b2640023521f5a35cd9a52fc8033d73a95d0d -> trunk/503b2640023521f5a35cd9a52fc8033d73a95d0d 2025-12-04T09:21:04.6569191Z * [new tag] trunk/518c2b1b3dab9a2ef2849e04b3bc2f20c1c41db9 -> trunk/518c2b1b3dab9a2ef2849e04b3bc2f20c1c41db9 2025-12-04T09:21:04.6570218Z * [new tag] trunk/5191b2fa68ba19960912bfd7fd721c79d76bb1f3 -> trunk/5191b2fa68ba19960912bfd7fd721c79d76bb1f3 2025-12-04T09:21:04.6571247Z * [new tag] trunk/52ac0f0dc4acacd219f1317fbc28ec631c01e07a -> trunk/52ac0f0dc4acacd219f1317fbc28ec631c01e07a 2025-12-04T09:21:04.6572268Z * [new tag] trunk/539ba711b029de9f191070f4f0d12f18f5b7f292 -> trunk/539ba711b029de9f191070f4f0d12f18f5b7f292 2025-12-04T09:21:04.6573295Z * [new tag] trunk/556375b55deebebbc56cb7aef81f4d52f031ba28 -> trunk/556375b55deebebbc56cb7aef81f4d52f031ba28 2025-12-04T09:21:04.6574308Z * [new tag] trunk/55c4ab554845481d0a69a3811937575fe8bb1a66 -> trunk/55c4ab554845481d0a69a3811937575fe8bb1a66 2025-12-04T09:21:04.6575297Z * [new tag] trunk/5634469fda9e5d98869c82c7d03bb08914245f96 -> trunk/5634469fda9e5d98869c82c7d03bb08914245f96 2025-12-04T09:21:04.6576298Z * [new tag] trunk/5778f6ff894686a975a9a23645178ae4c87ad5dc -> trunk/5778f6ff894686a975a9a23645178ae4c87ad5dc 2025-12-04T09:21:04.6577333Z * [new tag] trunk/587d63a3e07de5dc91065f9ef70bcacda9989068 -> trunk/587d63a3e07de5dc91065f9ef70bcacda9989068 2025-12-04T09:21:04.6578348Z * [new tag] trunk/597930f6b568852356ca9795dac76f9e4653adbd -> trunk/597930f6b568852356ca9795dac76f9e4653adbd 2025-12-04T09:21:04.6579364Z * [new tag] trunk/597df3a4e2a67b9fdbe1a89b2f4d74f822274db6 -> trunk/597df3a4e2a67b9fdbe1a89b2f4d74f822274db6 2025-12-04T09:21:04.6580379Z * [new tag] trunk/59abd50e931f4efb21b053f7a2911f5d8a49d883 -> trunk/59abd50e931f4efb21b053f7a2911f5d8a49d883 2025-12-04T09:21:04.6581403Z * [new tag] trunk/5a607febc04c3a2b5824c75f3f60307867439a2c -> trunk/5a607febc04c3a2b5824c75f3f60307867439a2c 2025-12-04T09:21:04.6582429Z * [new tag] trunk/5bf1cdf4755c54ef462b44cb8041b0a57311556b -> trunk/5bf1cdf4755c54ef462b44cb8041b0a57311556b 2025-12-04T09:21:04.6583444Z * [new tag] trunk/5f0030ba63d334d7e8c93a09e41403b89e4c573c -> trunk/5f0030ba63d334d7e8c93a09e41403b89e4c573c 2025-12-04T09:21:04.6584452Z * [new tag] trunk/5f21d27e71268464d362a96c9ac09ea475f7f202 -> trunk/5f21d27e71268464d362a96c9ac09ea475f7f202 2025-12-04T09:21:04.6585464Z * [new tag] trunk/5fafc13038c9988d9ac21fa793fbd5890604b447 -> trunk/5fafc13038c9988d9ac21fa793fbd5890604b447 2025-12-04T09:21:04.6586656Z * [new tag] trunk/61be54a31dc09b59d99b62176fb935aee0b924ef -> trunk/61be54a31dc09b59d99b62176fb935aee0b924ef 2025-12-04T09:21:04.6588025Z * [new tag] trunk/62d3ccd71484ed6a760d909b41487101bbc65719 -> trunk/62d3ccd71484ed6a760d909b41487101bbc65719 2025-12-04T09:21:04.6589066Z * [new tag] trunk/641cdb68ae27668eb441d0e49c87a0602c120c2b -> trunk/641cdb68ae27668eb441d0e49c87a0602c120c2b 2025-12-04T09:21:04.6590109Z * [new tag] trunk/65c4620d6bb0c6029f69762c22b91dda2294da9a -> trunk/65c4620d6bb0c6029f69762c22b91dda2294da9a 2025-12-04T09:21:04.6591211Z * [new tag] trunk/66004b993744b4106bf8afaba71f3c228a804206 -> trunk/66004b993744b4106bf8afaba71f3c228a804206 2025-12-04T09:21:04.6592344Z * [new tag] trunk/6658a04c7ca67acb64512341342e7b3ee13ee386 -> trunk/6658a04c7ca67acb64512341342e7b3ee13ee386 2025-12-04T09:21:04.6593391Z * [new tag] trunk/6864e309092a71f8ab0ca6a4dc7f8a4073fd31c4 -> trunk/6864e309092a71f8ab0ca6a4dc7f8a4073fd31c4 2025-12-04T09:21:04.6594507Z * [new tag] trunk/6c261c6cb07892c90ca19ed51c9705b1659a3f7d -> trunk/6c261c6cb07892c90ca19ed51c9705b1659a3f7d 2025-12-04T09:21:04.6595550Z * [new tag] trunk/6c8b6a043f1628188b6396b3a2a6e000ca68362b -> trunk/6c8b6a043f1628188b6396b3a2a6e000ca68362b 2025-12-04T09:21:04.6596601Z * [new tag] trunk/6ceb4a32f92ae67ce5d7d97931d17401ebf5ffa5 -> trunk/6ceb4a32f92ae67ce5d7d97931d17401ebf5ffa5 2025-12-04T09:21:04.6597652Z * [new tag] trunk/6e404e9b7d6f5fb0de86aa73888c3038248c17f8 -> trunk/6e404e9b7d6f5fb0de86aa73888c3038248c17f8 2025-12-04T09:21:04.6598727Z * [new tag] trunk/6ec30b490aee1db6bcdc7340abddef25784f08ec -> trunk/6ec30b490aee1db6bcdc7340abddef25784f08ec 2025-12-04T09:21:04.6599903Z * [new tag] trunk/6f2783a6c08e1db34275ff25176ffe9aebc30a71 -> trunk/6f2783a6c08e1db34275ff25176ffe9aebc30a71 2025-12-04T09:21:04.6601513Z * [new tag] trunk/6f53fefeb90ad3281119b5cfc4aa9ffd8a066e3d -> trunk/6f53fefeb90ad3281119b5cfc4aa9ffd8a066e3d 2025-12-04T09:21:04.6602604Z * [new tag] trunk/6f7dcf51e46d0c880db1a2f5c70de57adb576f4a -> trunk/6f7dcf51e46d0c880db1a2f5c70de57adb576f4a 2025-12-04T09:21:04.6603676Z * [new tag] trunk/6ff831180d2fa436c7f1c1af3adac641fce9d60e -> trunk/6ff831180d2fa436c7f1c1af3adac641fce9d60e 2025-12-04T09:21:04.6604742Z * [new tag] trunk/70076464a63ab218a7ceefb0e76ccd7131deb8f8 -> trunk/70076464a63ab218a7ceefb0e76ccd7131deb8f8 2025-12-04T09:21:04.6605793Z * [new tag] trunk/70d797a5fc109b20a517646fcaa819477cd0d485 -> trunk/70d797a5fc109b20a517646fcaa819477cd0d485 2025-12-04T09:21:04.6606851Z * [new tag] trunk/7348cb355ff0a6f79cd4871215aea72185748734 -> trunk/7348cb355ff0a6f79cd4871215aea72185748734 2025-12-04T09:21:04.6607891Z * [new tag] trunk/74fe26a1ebe32931783569f2e762e3c2c974901f -> trunk/74fe26a1ebe32931783569f2e762e3c2c974901f 2025-12-04T09:21:04.6608965Z * [new tag] trunk/76aeb8c7e0f795b3fddca134cbea9a69da3ee696 -> trunk/76aeb8c7e0f795b3fddca134cbea9a69da3ee696 2025-12-04T09:21:04.6610046Z * [new tag] trunk/7716da9fb23f27a65b41f9f016a2afadf281c18f -> trunk/7716da9fb23f27a65b41f9f016a2afadf281c18f 2025-12-04T09:21:04.6611102Z * [new tag] trunk/7741edd4ed665f3988052e260863efb508d61a03 -> trunk/7741edd4ed665f3988052e260863efb508d61a03 2025-12-04T09:21:04.6612141Z * [new tag] trunk/78adb3b3df41b45d2368b67226d2f864b78939a6 -> trunk/78adb3b3df41b45d2368b67226d2f864b78939a6 2025-12-04T09:21:04.6613322Z * [new tag] trunk/79d7b178225e5ed24d4e1db74e5abbff848f5fb7 -> trunk/79d7b178225e5ed24d4e1db74e5abbff848f5fb7 2025-12-04T09:21:04.6614342Z * [new tag] trunk/7a1e316115fc6996b3f2336822ba5d5f6179f0c3 -> trunk/7a1e316115fc6996b3f2336822ba5d5f6179f0c3 2025-12-04T09:21:04.6615357Z * [new tag] trunk/7a41b66367c38d0af3e8a90f7be48d6b281e7bca -> trunk/7a41b66367c38d0af3e8a90f7be48d6b281e7bca 2025-12-04T09:21:04.6616374Z * [new tag] trunk/7b7af390ea8541c611d1ce2018a6934188fc197b -> trunk/7b7af390ea8541c611d1ce2018a6934188fc197b 2025-12-04T09:21:04.6617386Z * [new tag] trunk/7ba4680f3755a560af81aa0f688791e367aa3609 -> trunk/7ba4680f3755a560af81aa0f688791e367aa3609 2025-12-04T09:21:04.6618415Z * [new tag] trunk/7bc2a66ded06a0b2549aa51d807edc5dc3e73d1b -> trunk/7bc2a66ded06a0b2549aa51d807edc5dc3e73d1b 2025-12-04T09:21:04.6619455Z * [new tag] trunk/7c648509a7470ace9fb2bae960dd4790f7e943e9 -> trunk/7c648509a7470ace9fb2bae960dd4790f7e943e9 2025-12-04T09:21:04.6620619Z * [new tag] trunk/7cbc2d034cecd21ab5c9707d0a9c525c17143fb8 -> trunk/7cbc2d034cecd21ab5c9707d0a9c525c17143fb8 2025-12-04T09:21:04.6621660Z * [new tag] trunk/7d1bbaf4ba301ea3fba6f3c7bc02d58f6417aaed -> trunk/7d1bbaf4ba301ea3fba6f3c7bc02d58f6417aaed 2025-12-04T09:21:04.6622961Z * [new tag] trunk/7d2a33e4ebf60b217a3cd77feae19231eb996fc8 -> trunk/7d2a33e4ebf60b217a3cd77feae19231eb996fc8 2025-12-04T09:21:04.6623989Z * [new tag] trunk/7eb625920054b1126a7d2d99818aaa188c6ba95e -> trunk/7eb625920054b1126a7d2d99818aaa188c6ba95e 2025-12-04T09:21:04.6625022Z * [new tag] trunk/7f55ba19c456a3d6cc443dd9edb6bb7cca677ead -> trunk/7f55ba19c456a3d6cc443dd9edb6bb7cca677ead 2025-12-04T09:21:04.6626029Z * [new tag] trunk/81af382128efa094d8702e18f2c133760904c718 -> trunk/81af382128efa094d8702e18f2c133760904c718 2025-12-04T09:21:04.6627294Z * [new tag] trunk/84149583d483e9c973c9a0feda70e4f3964947b0 -> trunk/84149583d483e9c973c9a0feda70e4f3964947b0 2025-12-04T09:21:04.6628347Z * [new tag] trunk/85a315917efe82c24306be805c584ec044951c75 -> trunk/85a315917efe82c24306be805c584ec044951c75 2025-12-04T09:21:04.6629395Z * [new tag] trunk/87329491c82a5f8c1cc4ec11d8f55a5de2551ece -> trunk/87329491c82a5f8c1cc4ec11d8f55a5de2551ece 2025-12-04T09:21:04.6630446Z * [new tag] trunk/892640e25aeefa8007c5af837214b4502b6b62a6 -> trunk/892640e25aeefa8007c5af837214b4502b6b62a6 2025-12-04T09:21:04.6631502Z * [new tag] trunk/89e3bbcb5b5321dc8b9520b4d5a8ee60cea1d0b4 -> trunk/89e3bbcb5b5321dc8b9520b4d5a8ee60cea1d0b4 2025-12-04T09:21:04.6632568Z * [new tag] trunk/8c73bbbb02159223c0c97d268a0a74cb78158a1c -> trunk/8c73bbbb02159223c0c97d268a0a74cb78158a1c 2025-12-04T09:21:04.6633640Z * [new tag] trunk/8d56e98c8db988a22cb2dfaeefb30bc7d2a3cc43 -> trunk/8d56e98c8db988a22cb2dfaeefb30bc7d2a3cc43 2025-12-04T09:21:04.6634703Z * [new tag] trunk/8d9dd9603e5ee26c01007f0cd4f018e584840922 -> trunk/8d9dd9603e5ee26c01007f0cd4f018e584840922 2025-12-04T09:21:04.6635762Z * [new tag] trunk/8ef0c0b02b062d75e7c9be2594914a3e784d23ca -> trunk/8ef0c0b02b062d75e7c9be2594914a3e784d23ca 2025-12-04T09:21:04.6636819Z * [new tag] trunk/90b27e7e8352cde97d32ddad24740ef819633f38 -> trunk/90b27e7e8352cde97d32ddad24740ef819633f38 2025-12-04T09:21:04.6637857Z * [new tag] trunk/90f0139e64b2951815d524b6a373bed20c4fbf90 -> trunk/90f0139e64b2951815d524b6a373bed20c4fbf90 2025-12-04T09:21:04.6639002Z * [new tag] trunk/93d0d6838c56af59b0dba794e6aa08f0c1c7799c -> trunk/93d0d6838c56af59b0dba794e6aa08f0c1c7799c 2025-12-04T09:21:04.6640031Z * [new tag] trunk/94ca8d5f1e81fea3ae488650a0fb6795049a9f87 -> trunk/94ca8d5f1e81fea3ae488650a0fb6795049a9f87 2025-12-04T09:21:04.6641071Z * [new tag] trunk/9844fbeadd5cebdf1281d6fbf79164139c352693 -> trunk/9844fbeadd5cebdf1281d6fbf79164139c352693 2025-12-04T09:21:04.6642107Z * [new tag] trunk/99024dec888ec1e50b546822a32b6fb2f35e5eaa -> trunk/99024dec888ec1e50b546822a32b6fb2f35e5eaa 2025-12-04T09:21:04.6643135Z * [new tag] trunk/9a296e640fc88aa44d275b48cd9cc30c573b169d -> trunk/9a296e640fc88aa44d275b48cd9cc30c573b169d 2025-12-04T09:21:04.6644152Z * [new tag] trunk/9b3e34d8589b29f7b4e7fab6f78711b7ca6e4639 -> trunk/9b3e34d8589b29f7b4e7fab6f78711b7ca6e4639 2025-12-04T09:21:04.6645171Z * [new tag] trunk/9cd055e547e9b67a5f9827f8999c38d7eda1bcb8 -> trunk/9cd055e547e9b67a5f9827f8999c38d7eda1bcb8 2025-12-04T09:21:04.6646190Z * [new tag] trunk/9f0df5686cb4ada94f94620acba2e3c3f363b11d -> trunk/9f0df5686cb4ada94f94620acba2e3c3f363b11d 2025-12-04T09:21:04.6647224Z * [new tag] trunk/9f7fceb887d0cfa0326a59b887821c63ff11340a -> trunk/9f7fceb887d0cfa0326a59b887821c63ff11340a 2025-12-04T09:21:04.6648228Z * [new tag] trunk/9f8ef8855d3078d70f7b782540ff2aaf158d6742 -> trunk/9f8ef8855d3078d70f7b782540ff2aaf158d6742 2025-12-04T09:21:04.6649809Z * [new tag] trunk/9fb52efc797b47a1f425a03aa5e47b866d8b1098 -> trunk/9fb52efc797b47a1f425a03aa5e47b866d8b1098 2025-12-04T09:21:04.6650885Z * [new tag] trunk/9ff4a2ebc5762d46c73e46b1b523d7ff349fedfa -> trunk/9ff4a2ebc5762d46c73e46b1b523d7ff349fedfa 2025-12-04T09:21:04.6652015Z * [new tag] trunk/a0f3937b94422354538ebbd47202d5b0e8a3fd0d -> trunk/a0f3937b94422354538ebbd47202d5b0e8a3fd0d 2025-12-04T09:21:04.6653053Z * [new tag] trunk/a15066c28b3145e6edbfc88359d0411d14cfc70c -> trunk/a15066c28b3145e6edbfc88359d0411d14cfc70c 2025-12-04T09:21:04.6654159Z * [new tag] trunk/a20f775e82564d2a9979221ed7f3b8d7cf54ce90 -> trunk/a20f775e82564d2a9979221ed7f3b8d7cf54ce90 2025-12-04T09:21:04.6655217Z * [new tag] trunk/a2973fb00ec002dd4b6bbf07385f066efb259b8c -> trunk/a2973fb00ec002dd4b6bbf07385f066efb259b8c 2025-12-04T09:21:04.6656282Z * [new tag] trunk/a7dc6dab9ad911259d4801c502907e531594db45 -> trunk/a7dc6dab9ad911259d4801c502907e531594db45 2025-12-04T09:21:04.6657341Z * [new tag] trunk/a951a9cee65c01660bbc6e6fded90ecb10fa6109 -> trunk/a951a9cee65c01660bbc6e6fded90ecb10fa6109 2025-12-04T09:21:04.6658420Z * [new tag] trunk/abfa1a6d65c7c159e35c72c25979b9da4971689e -> trunk/abfa1a6d65c7c159e35c72c25979b9da4971689e 2025-12-04T09:21:04.6659472Z * [new tag] trunk/ae3a2395bf66151078e2d201716f7d63ce1c6f3e -> trunk/ae3a2395bf66151078e2d201716f7d63ce1c6f3e 2025-12-04T09:21:04.6660545Z * [new tag] trunk/afdff7f0325080dedac44d080cb5a3b0e65e6c5e -> trunk/afdff7f0325080dedac44d080cb5a3b0e65e6c5e 2025-12-04T09:21:04.6661626Z * [new tag] trunk/b1aed4e7a72c03a38f44543aaea0dae2e9b76d48 -> trunk/b1aed4e7a72c03a38f44543aaea0dae2e9b76d48 2025-12-04T09:21:04.6662696Z * [new tag] trunk/b1decff555cd50e2123c8c6e25cc0d447c411f62 -> trunk/b1decff555cd50e2123c8c6e25cc0d447c411f62 2025-12-04T09:21:04.6663766Z * [new tag] trunk/b2b6b034c9fd08672c40e63ef243556ad4c49bd2 -> trunk/b2b6b034c9fd08672c40e63ef243556ad4c49bd2 2025-12-04T09:21:04.6664231Z * [new tag] trunk/b39813b4a04931682b0491adba2138d01d716d99 -> trunk/b39813b4a04931682b0491adba2138d01d716d99 2025-12-04T09:21:04.6664701Z * [new tag] trunk/b3a7edb2311367974cc7cd764cfb11a5d6758b24 -> trunk/b3a7edb2311367974cc7cd764cfb11a5d6758b24 2025-12-04T09:21:04.6665171Z * [new tag] trunk/b4cc1329c86acaef6d42c1fac7169b8d870ab0d7 -> trunk/b4cc1329c86acaef6d42c1fac7169b8d870ab0d7 2025-12-04T09:21:04.6665633Z * [new tag] trunk/b555c39217f765759954a4f9f9bd1e9b87bed11a -> trunk/b555c39217f765759954a4f9f9bd1e9b87bed11a 2025-12-04T09:21:04.6666088Z * [new tag] trunk/b6b6c80379388b7f9932c3e6a0f9907bf430e417 -> trunk/b6b6c80379388b7f9932c3e6a0f9907bf430e417 2025-12-04T09:21:04.6666653Z * [new tag] trunk/b6b6d912df0b6f4082f8e50b18bd1de1dd7325f4 -> trunk/b6b6d912df0b6f4082f8e50b18bd1de1dd7325f4 2025-12-04T09:21:04.6667123Z * [new tag] trunk/b7d60685f8cbc939b68a20871e90db67e729329b -> trunk/b7d60685f8cbc939b68a20871e90db67e729329b 2025-12-04T09:21:04.6667595Z * [new tag] trunk/b7f6b9a4fc6259f7af068f31868b3119bb1bac3e -> trunk/b7f6b9a4fc6259f7af068f31868b3119bb1bac3e 2025-12-04T09:21:04.6668059Z * [new tag] trunk/b8c4ba3593761e7b2a3ebd86f040fb07b47c02cf -> trunk/b8c4ba3593761e7b2a3ebd86f040fb07b47c02cf 2025-12-04T09:21:04.6668517Z * [new tag] trunk/b9c8f3a4884befb965ff42620ce44a71b04887f5 -> trunk/b9c8f3a4884befb965ff42620ce44a71b04887f5 2025-12-04T09:21:04.6668980Z * [new tag] trunk/ba1412546f3082c0958c077acc2025e4dbc33f1f -> trunk/ba1412546f3082c0958c077acc2025e4dbc33f1f 2025-12-04T09:21:04.6669453Z * [new tag] trunk/bac403c0b38c63bdbcc0c31f1c2b0bc0260f610f -> trunk/bac403c0b38c63bdbcc0c31f1c2b0bc0260f610f 2025-12-04T09:21:04.6669975Z * [new tag] trunk/bb3034198b459401fabeab254e1b99f0115046e2 -> trunk/bb3034198b459401fabeab254e1b99f0115046e2 2025-12-04T09:21:04.6670446Z * [new tag] trunk/bc39b2b3bc7a6e19a42e62bd576974035086fe55 -> trunk/bc39b2b3bc7a6e19a42e62bd576974035086fe55 2025-12-04T09:21:04.6670967Z * [new tag] trunk/bc43d5b297f207a11d83d77ddf0152bdaabe15a8 -> trunk/bc43d5b297f207a11d83d77ddf0152bdaabe15a8 2025-12-04T09:21:04.6671437Z * [new tag] trunk/bc6a4863c7246a6493d16d4ea6eee71ec07c6a09 -> trunk/bc6a4863c7246a6493d16d4ea6eee71ec07c6a09 2025-12-04T09:21:04.6671911Z * [new tag] trunk/bea4912944defdbcb8b061800caab6cbbbd01df5 -> trunk/bea4912944defdbcb8b061800caab6cbbbd01df5 2025-12-04T09:21:04.6672390Z * [new tag] trunk/c04e2c656f48d82d1521b867bbbf03967b9b7564 -> trunk/c04e2c656f48d82d1521b867bbbf03967b9b7564 2025-12-04T09:21:04.6672843Z * [new tag] trunk/c0660bcee27e7d7731634e274576a7081882bede -> trunk/c0660bcee27e7d7731634e274576a7081882bede 2025-12-04T09:21:04.6673323Z * [new tag] trunk/c178ed43d3d99cbefe84fbfb21d6f282b20d62ac -> trunk/c178ed43d3d99cbefe84fbfb21d6f282b20d62ac 2025-12-04T09:21:04.6673778Z * [new tag] trunk/c55b1e8f61d041ee436d697449eb028931d574fb -> trunk/c55b1e8f61d041ee436d697449eb028931d574fb 2025-12-04T09:21:04.6674237Z * [new tag] trunk/c6ae7579fe12fe75f1a8f7043a494c90567273f1 -> trunk/c6ae7579fe12fe75f1a8f7043a494c90567273f1 2025-12-04T09:21:04.6674711Z * [new tag] trunk/c8210e7d94bad5ae21ac389fa4ba8a463c76c4d0 -> trunk/c8210e7d94bad5ae21ac389fa4ba8a463c76c4d0 2025-12-04T09:21:04.6675154Z * [new tag] trunk/cc0853af42122f8185321f542616f4474e717f09 -> trunk/cc0853af42122f8185321f542616f4474e717f09 2025-12-04T09:21:04.6675623Z * [new tag] trunk/cddec6562eabfa390d014fa3741a5659cf9c94c9 -> trunk/cddec6562eabfa390d014fa3741a5659cf9c94c9 2025-12-04T09:21:04.6676101Z * [new tag] trunk/ce5e7e3bf1f4b69a4f4f93d288ba75b906df492a -> trunk/ce5e7e3bf1f4b69a4f4f93d288ba75b906df492a 2025-12-04T09:21:04.6676568Z * [new tag] trunk/d038b0130ec7c20ebcac219301292fd8e98a1ace -> trunk/d038b0130ec7c20ebcac219301292fd8e98a1ace 2025-12-04T09:21:04.6677042Z * [new tag] trunk/d16447dacaf2420ea175f0c275c75da951f57d39 -> trunk/d16447dacaf2420ea175f0c275c75da951f57d39 2025-12-04T09:21:04.6677496Z * [new tag] trunk/d19f1e8cab6810bb2e99141f9976665954c67a50 -> trunk/d19f1e8cab6810bb2e99141f9976665954c67a50 2025-12-04T09:21:04.6677959Z * [new tag] trunk/d1c9f03b2a5af4104721712f8cdffe9b4f340c01 -> trunk/d1c9f03b2a5af4104721712f8cdffe9b4f340c01 2025-12-04T09:21:04.6678539Z * [new tag] trunk/d40f4950f2b7f7aa380a22fe0f6166e71680fbcf -> trunk/d40f4950f2b7f7aa380a22fe0f6166e71680fbcf 2025-12-04T09:21:04.6678984Z * [new tag] trunk/d5038950bacfe36bbf24a47a455fe76901deb8e8 -> trunk/d5038950bacfe36bbf24a47a455fe76901deb8e8 2025-12-04T09:21:04.6679448Z * [new tag] trunk/d54ff42903c2ae0533931ff11d23b35f875bdb3d -> trunk/d54ff42903c2ae0533931ff11d23b35f875bdb3d 2025-12-04T09:21:04.6680412Z * [new tag] trunk/d76697633a2d2b9cced1ae21161849b33bfe7e47 -> trunk/d76697633a2d2b9cced1ae21161849b33bfe7e47 2025-12-04T09:21:04.6681340Z * [new tag] trunk/d78f52b199c547106d4cd9d2856dd0805c118bf1 -> trunk/d78f52b199c547106d4cd9d2856dd0805c118bf1 2025-12-04T09:21:04.6682327Z * [new tag] trunk/d8fd5c6eed28e5004150691d048a3f6785e19a8e -> trunk/d8fd5c6eed28e5004150691d048a3f6785e19a8e 2025-12-04T09:21:04.6683326Z * [new tag] trunk/d900f5e86745dec76713f4b0ef07005ef36b2f5a -> trunk/d900f5e86745dec76713f4b0ef07005ef36b2f5a 2025-12-04T09:21:04.6684343Z * [new tag] trunk/d973dc6b87d763859fe1c5bd1287e3b6b1c49d1b -> trunk/d973dc6b87d763859fe1c5bd1287e3b6b1c49d1b 2025-12-04T09:21:04.6685481Z * [new tag] trunk/d998c03304cb6ede76e1ed535b4ddeb6c2bf40ec -> trunk/d998c03304cb6ede76e1ed535b4ddeb6c2bf40ec 2025-12-04T09:21:04.6686552Z * [new tag] trunk/d9cb8a70833101dbbe16b99520cfbdd70d0a87bf -> trunk/d9cb8a70833101dbbe16b99520cfbdd70d0a87bf 2025-12-04T09:21:04.6687475Z * [new tag] trunk/d9d5e91b43f70eb8637af55db6856d49be391ffd -> trunk/d9d5e91b43f70eb8637af55db6856d49be391ffd 2025-12-04T09:21:04.6688486Z * [new tag] trunk/dd18a75336a4fbd7497955cc5665904724fce889 -> trunk/dd18a75336a4fbd7497955cc5665904724fce889 2025-12-04T09:21:04.6689462Z * [new tag] trunk/ded9bcd61a059bf723e6e84689552962b480ea77 -> trunk/ded9bcd61a059bf723e6e84689552962b480ea77 2025-12-04T09:21:04.6690883Z * [new tag] trunk/dfbd3714d15c37a7b83b322a6b60f997fc00f50c -> trunk/dfbd3714d15c37a7b83b322a6b60f997fc00f50c 2025-12-04T09:21:04.6691883Z * [new tag] trunk/e115f9f4e4b039f8e9a642aaa2bd8254a920541b -> trunk/e115f9f4e4b039f8e9a642aaa2bd8254a920541b 2025-12-04T09:21:04.6692724Z * [new tag] trunk/e3f24fd73ad74c6e7176687986436956c7c18235 -> trunk/e3f24fd73ad74c6e7176687986436956c7c18235 2025-12-04T09:21:04.6693871Z * [new tag] trunk/e7d24d3ff93d1503ba63860b7057438ad93f918e -> trunk/e7d24d3ff93d1503ba63860b7057438ad93f918e 2025-12-04T09:21:04.6694881Z * [new tag] trunk/ea7035f462a0d2830865ee86c832bd101e1427fc -> trunk/ea7035f462a0d2830865ee86c832bd101e1427fc 2025-12-04T09:21:04.6695944Z * [new tag] trunk/eabb7ad2128580ef674446027b95bcf4e21e8df3 -> trunk/eabb7ad2128580ef674446027b95bcf4e21e8df3 2025-12-04T09:21:04.6697002Z * [new tag] trunk/eb5c63652a33da42e7018c23df5f20a3eb4c6ccf -> trunk/eb5c63652a33da42e7018c23df5f20a3eb4c6ccf 2025-12-04T09:21:04.6698000Z * [new tag] trunk/ec2c71f5c85021b8938cdafadce24c15a36fd93e -> trunk/ec2c71f5c85021b8938cdafadce24c15a36fd93e 2025-12-04T09:21:04.6699026Z * [new tag] trunk/ecbcc3f6bf327856b435b259ac63cc2f328c4b4e -> trunk/ecbcc3f6bf327856b435b259ac63cc2f328c4b4e 2025-12-04T09:21:04.6700543Z * [new tag] trunk/ee87bbe876c42575e961b32a0827d76bc9782ca2 -> trunk/ee87bbe876c42575e961b32a0827d76bc9782ca2 2025-12-04T09:21:04.6701996Z * [new tag] trunk/ef019d1d431c4c5a95b594cb90d40a50cd00f5e4 -> trunk/ef019d1d431c4c5a95b594cb90d40a50cd00f5e4 2025-12-04T09:21:04.6703409Z * [new tag] trunk/ef8ecc13830a86c4b231f1aad9aba7851db61b53 -> trunk/ef8ecc13830a86c4b231f1aad9aba7851db61b53 2025-12-04T09:21:04.6704344Z * [new tag] trunk/f1076f5510920044912247b1abb8760cb820f598 -> trunk/f1076f5510920044912247b1abb8760cb820f598 2025-12-04T09:21:04.6705394Z * [new tag] trunk/f2d6a75a00a1d648ca9a0abc6a33e14c3dea6c40 -> trunk/f2d6a75a00a1d648ca9a0abc6a33e14c3dea6c40 2025-12-04T09:21:04.6706393Z * [new tag] trunk/f47dd0ddef1359e5b43e4b962412f67b30ecde56 -> trunk/f47dd0ddef1359e5b43e4b962412f67b30ecde56 2025-12-04T09:21:04.6707586Z * [new tag] trunk/f49d32dfa4730dcfb1b60eeeb369b5889da983c8 -> trunk/f49d32dfa4730dcfb1b60eeeb369b5889da983c8 2025-12-04T09:21:04.6711344Z * [new tag] trunk/f4dedf78fc30fd4b93975787ca6074ee89db9467 -> trunk/f4dedf78fc30fd4b93975787ca6074ee89db9467 2025-12-04T09:21:04.6711862Z * [new tag] trunk/f7c0d03819ebed05c4038f095d66d1b8c54aca17 -> trunk/f7c0d03819ebed05c4038f095d66d1b8c54aca17 2025-12-04T09:21:04.6717305Z * [new tag] trunk/f7e1bd80a063e17453c361837ba6ea2570920a73 -> trunk/f7e1bd80a063e17453c361837ba6ea2570920a73 2025-12-04T09:21:04.6717762Z * [new tag] trunk/f9bd6c53624c7c0ea3772de78498326e84c2f0e7 -> trunk/f9bd6c53624c7c0ea3772de78498326e84c2f0e7 2025-12-04T09:21:04.6718238Z * [new tag] trunk/fb5be221a46b51bfc9509013b0d85bc5a9d4f15b -> trunk/fb5be221a46b51bfc9509013b0d85bc5a9d4f15b 2025-12-04T09:21:04.6718711Z * [new tag] trunk/fdf863d5e1de3b2688c9511e96876e34581dbfd7 -> trunk/fdf863d5e1de3b2688c9511e96876e34581dbfd7 2025-12-04T09:21:04.6719327Z * [new tag] trunk/fe0e65adfc0e7ca6e5f57e6ea8b16bd5cc967307 -> trunk/fe0e65adfc0e7ca6e5f57e6ea8b16bd5cc967307 2025-12-04T09:21:04.6719795Z * [new tag] trunk/fec710bf89173f5355468a7ce1afe9157c3d9009 -> trunk/fec710bf89173f5355468a7ce1afe9157c3d9009 2025-12-04T09:21:04.6720358Z * [new tag] trunk/ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 -> trunk/ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:21:04.6720541Z * [new tag] v0.1.1 -> v0.1.1 2025-12-04T09:21:04.6720714Z * [new tag] v0.1.10 -> v0.1.10 2025-12-04T09:21:04.6720874Z * [new tag] v0.1.11 -> v0.1.11 2025-12-04T09:21:04.6721384Z * [new tag] v0.1.12 -> v0.1.12 2025-12-04T09:21:04.6722260Z * [new tag] v0.1.2 -> v0.1.2 2025-12-04T09:21:04.6723127Z * [new tag] v0.1.3 -> v0.1.3 2025-12-04T09:21:04.6724106Z * [new tag] v0.1.4 -> v0.1.4 2025-12-04T09:21:04.6725063Z * [new tag] v0.1.5 -> v0.1.5 2025-12-04T09:21:04.6725957Z * [new tag] v0.1.6 -> v0.1.6 2025-12-04T09:21:04.6726756Z * [new tag] v0.1.7 -> v0.1.7 2025-12-04T09:21:04.6727651Z * [new tag] v0.1.8 -> v0.1.8 2025-12-04T09:21:04.6728536Z * [new tag] v0.1.9 -> v0.1.9 2025-12-04T09:21:04.6729502Z * [new tag] v0.2.0 -> v0.2.0 2025-12-04T09:21:04.6730458Z * [new tag] v0.3.0 -> v0.3.0 2025-12-04T09:21:04.6731487Z * [new tag] v0.3.1 -> v0.3.1 2025-12-04T09:21:04.6732417Z * [new tag] v0.4.0 -> v0.4.0 2025-12-04T09:21:04.6733231Z * [new tag] v0.4.1 -> v0.4.1 2025-12-04T09:21:04.6734193Z * [new tag] v1.0.0 -> v1.0.0 2025-12-04T09:21:04.6735169Z * [new tag] v1.0.0a0 -> v1.0.0a0 2025-12-04T09:21:04.6736076Z * [new tag] v1.0.1 -> v1.0.1 2025-12-04T09:21:04.6737039Z * [new tag] v1.0rc0 -> v1.0rc0 2025-12-04T09:21:04.6738250Z * [new tag] v1.0rc1 -> v1.0rc1 2025-12-04T09:21:04.6739219Z * [new tag] v1.1.0 -> v1.1.0 2025-12-04T09:21:04.6740162Z * [new tag] v1.1.0a0 -> v1.1.0a0 2025-12-04T09:21:04.6741299Z * [new tag] v1.10.0 -> v1.10.0 2025-12-04T09:21:04.6742309Z * [new tag] v1.10.0-rc1 -> v1.10.0-rc1 2025-12-04T09:21:04.6743259Z * [new tag] v1.10.0-rc2 -> v1.10.0-rc2 2025-12-04T09:21:04.6743998Z * [new tag] v1.10.0-rc3 -> v1.10.0-rc3 2025-12-04T09:21:04.6745034Z * [new tag] v1.10.1 -> v1.10.1 2025-12-04T09:21:04.6746037Z * [new tag] v1.10.1-rc1 -> v1.10.1-rc1 2025-12-04T09:21:04.6746848Z * [new tag] v1.10.2 -> v1.10.2 2025-12-04T09:21:04.6748211Z * [new tag] v1.10.2-rc1 -> v1.10.2-rc1 2025-12-04T09:21:04.6748849Z * [new tag] v1.11.0 -> v1.11.0 2025-12-04T09:21:04.6750098Z * [new tag] v1.11.0-rc1 -> v1.11.0-rc1 2025-12-04T09:21:04.6751300Z * [new tag] v1.11.0-rc2 -> v1.11.0-rc2 2025-12-04T09:21:04.6752345Z * [new tag] v1.11.0-rc3 -> v1.11.0-rc3 2025-12-04T09:21:04.6753333Z * [new tag] v1.11.0-rc4 -> v1.11.0-rc4 2025-12-04T09:21:04.6754287Z * [new tag] v1.11.0-rc5 -> v1.11.0-rc5 2025-12-04T09:21:04.6755059Z * [new tag] v1.11.0-rc6 -> v1.11.0-rc6 2025-12-04T09:21:04.6755821Z * [new tag] v1.11.0-rc7 -> v1.11.0-rc7 2025-12-04T09:21:04.6756981Z * [new tag] v1.12.0 -> v1.12.0 2025-12-04T09:21:04.6757826Z * [new tag] v1.12.0-rc1 -> v1.12.0-rc1 2025-12-04T09:21:04.6758987Z * [new tag] v1.12.0-rc2 -> v1.12.0-rc2 2025-12-04T09:21:04.6759984Z * [new tag] v1.12.0-rc3 -> v1.12.0-rc3 2025-12-04T09:21:04.6760975Z * [new tag] v1.12.0-rc4 -> v1.12.0-rc4 2025-12-04T09:21:04.6761969Z * [new tag] v1.12.0-rc5 -> v1.12.0-rc5 2025-12-04T09:21:04.6763051Z * [new tag] v1.12.0-rc6 -> v1.12.0-rc6 2025-12-04T09:21:04.6763762Z * [new tag] v1.12.0-rc7 -> v1.12.0-rc7 2025-12-04T09:21:04.6764495Z * [new tag] v1.12.0-rc8 -> v1.12.0-rc8 2025-12-04T09:21:04.6765223Z * [new tag] v1.12.1 -> v1.12.1 2025-12-04T09:21:04.6766346Z * [new tag] v1.12.1-rc1 -> v1.12.1-rc1 2025-12-04T09:21:04.6767295Z * [new tag] v1.12.1-rc2 -> v1.12.1-rc2 2025-12-04T09:21:04.6768342Z * [new tag] v1.12.1-rc3 -> v1.12.1-rc3 2025-12-04T09:21:04.6769335Z * [new tag] v1.12.1-rc4 -> v1.12.1-rc4 2025-12-04T09:21:04.6770026Z * [new tag] v1.12.1-rc5 -> v1.12.1-rc5 2025-12-04T09:21:04.6771070Z * [new tag] v1.13.0 -> v1.13.0 2025-12-04T09:21:04.6772033Z * [new tag] v1.13.0-rc1 -> v1.13.0-rc1 2025-12-04T09:21:04.6773013Z * [new tag] v1.13.0-rc2 -> v1.13.0-rc2 2025-12-04T09:21:04.6779080Z * [new tag] v1.13.0-rc3 -> v1.13.0-rc3 2025-12-04T09:21:04.6780400Z * [new tag] v1.13.0-rc4 -> v1.13.0-rc4 2025-12-04T09:21:04.6781093Z * [new tag] v1.13.0-rc5 -> v1.13.0-rc5 2025-12-04T09:21:04.6781857Z * [new tag] v1.13.0-rc6 -> v1.13.0-rc6 2025-12-04T09:21:04.6782991Z * [new tag] v1.13.1 -> v1.13.1 2025-12-04T09:21:04.6783716Z * [new tag] v1.13.1-rc1 -> v1.13.1-rc1 2025-12-04T09:21:04.6784728Z * [new tag] v1.2.0 -> v1.2.0 2025-12-04T09:21:04.6785699Z * [new tag] v1.2.0a0 -> v1.2.0a0 2025-12-04T09:21:04.6786902Z * [new tag] v1.3.0 -> v1.3.0 2025-12-04T09:21:04.6787897Z * [new tag] v1.3.0a0 -> v1.3.0a0 2025-12-04T09:21:04.6788606Z * [new tag] v1.3.1 -> v1.3.1 2025-12-04T09:21:04.6789608Z * [new tag] v1.4.0 -> v1.4.0 2025-12-04T09:21:04.6790518Z * [new tag] v1.4.0a0 -> v1.4.0a0 2025-12-04T09:21:04.6791273Z * [new tag] v1.4.1 -> v1.4.1 2025-12-04T09:21:04.6792389Z * [new tag] v1.5.0 -> v1.5.0 2025-12-04T09:21:04.6793452Z * [new tag] v1.5.0-rc1 -> v1.5.0-rc1 2025-12-04T09:21:04.6794463Z * [new tag] v1.5.0-rc2 -> v1.5.0-rc2 2025-12-04T09:21:04.6795499Z * [new tag] v1.5.0-rc3 -> v1.5.0-rc3 2025-12-04T09:21:04.6796444Z * [new tag] v1.5.0-rc4 -> v1.5.0-rc4 2025-12-04T09:21:04.6797156Z * [new tag] v1.5.0-rc5 -> v1.5.0-rc5 2025-12-04T09:21:04.6798430Z * [new tag] v1.5.1 -> v1.5.1 2025-12-04T09:21:04.6799129Z * [new tag] v1.5.1-rc1 -> v1.5.1-rc1 2025-12-04T09:21:04.6799872Z * [new tag] v1.6.0 -> v1.6.0 2025-12-04T09:21:04.6801497Z * [new tag] v1.6.0-rc1 -> v1.6.0-rc1 2025-12-04T09:21:04.6804472Z * [new tag] v1.6.0-rc2 -> v1.6.0-rc2 2025-12-04T09:21:04.6805321Z * [new tag] v1.6.0-rc3 -> v1.6.0-rc3 2025-12-04T09:21:04.6806318Z * [new tag] v1.6.0-rc4 -> v1.6.0-rc4 2025-12-04T09:21:04.6807731Z * [new tag] v1.6.0-rc5 -> v1.6.0-rc5 2025-12-04T09:21:04.6808752Z * [new tag] v1.6.0-rc6 -> v1.6.0-rc6 2025-12-04T09:21:04.6809457Z * [new tag] v1.6.0-rc7 -> v1.6.0-rc7 2025-12-04T09:21:04.6810567Z * [new tag] v1.7.0 -> v1.7.0 2025-12-04T09:21:04.6811570Z * [new tag] v1.7.0-rc1 -> v1.7.0-rc1 2025-12-04T09:21:04.6812644Z * [new tag] v1.7.0-rc2 -> v1.7.0-rc2 2025-12-04T09:21:04.6813781Z * [new tag] v1.7.0-rc3 -> v1.7.0-rc3 2025-12-04T09:21:04.6814652Z * [new tag] v1.7.0-rc4 -> v1.7.0-rc4 2025-12-04T09:21:04.6815447Z * [new tag] v1.7.1 -> v1.7.1 2025-12-04T09:21:04.6816588Z * [new tag] v1.7.1-rc1 -> v1.7.1-rc1 2025-12-04T09:21:04.6817637Z * [new tag] v1.7.1-rc2 -> v1.7.1-rc2 2025-12-04T09:21:04.6818361Z * [new tag] v1.7.1-rc3 -> v1.7.1-rc3 2025-12-04T09:21:04.6819364Z * [new tag] v1.8.0 -> v1.8.0 2025-12-04T09:21:04.6820183Z * [new tag] v1.8.0-rc1 -> v1.8.0-rc1 2025-12-04T09:21:04.6821242Z * [new tag] v1.8.0-rc2 -> v1.8.0-rc2 2025-12-04T09:21:04.6822218Z * [new tag] v1.8.0-rc3 -> v1.8.0-rc3 2025-12-04T09:21:04.6823220Z * [new tag] v1.8.0-rc4 -> v1.8.0-rc4 2025-12-04T09:21:04.6823790Z * [new tag] v1.8.0-rc5 -> v1.8.0-rc5 2025-12-04T09:21:04.6824551Z * [new tag] v1.8.1 -> v1.8.1 2025-12-04T09:21:04.6825567Z * [new tag] v1.8.1-rc1 -> v1.8.1-rc1 2025-12-04T09:21:04.6826282Z * [new tag] v1.8.1-rc2 -> v1.8.1-rc2 2025-12-04T09:21:04.6827375Z * [new tag] v1.8.1-rc3 -> v1.8.1-rc3 2025-12-04T09:21:04.6828980Z * [new tag] v1.8.2 -> v1.8.2 2025-12-04T09:21:04.6829737Z * [new tag] v1.8.2-rc1 -> v1.8.2-rc1 2025-12-04T09:21:04.6830805Z * [new tag] v1.9.0 -> v1.9.0 2025-12-04T09:21:04.6831788Z * [new tag] v1.9.0-rc1 -> v1.9.0-rc1 2025-12-04T09:21:04.6832852Z * [new tag] v1.9.0-rc2 -> v1.9.0-rc2 2025-12-04T09:21:04.6833880Z * [new tag] v1.9.0-rc3 -> v1.9.0-rc3 2025-12-04T09:21:04.6834622Z * [new tag] v1.9.0-rc4 -> v1.9.0-rc4 2025-12-04T09:21:04.6835657Z * [new tag] v1.9.1 -> v1.9.1 2025-12-04T09:21:04.6841632Z * [new tag] v1.9.1-rc1 -> v1.9.1-rc1 2025-12-04T09:21:04.6842687Z * [new tag] v1.9.1-rc2 -> v1.9.1-rc2 2025-12-04T09:21:04.6843759Z * [new tag] v2.0.0 -> v2.0.0 2025-12-04T09:21:04.6844745Z * [new tag] v2.0.0-rc1 -> v2.0.0-rc1 2025-12-04T09:21:04.6845778Z * [new tag] v2.0.0-rc2 -> v2.0.0-rc2 2025-12-04T09:21:04.6846883Z * [new tag] v2.0.0-rc3 -> v2.0.0-rc3 2025-12-04T09:21:04.6847851Z * [new tag] v2.0.0-rc4 -> v2.0.0-rc4 2025-12-04T09:21:04.6848986Z * [new tag] v2.0.0-rc5 -> v2.0.0-rc5 2025-12-04T09:21:04.6849756Z * [new tag] v2.0.0-rc6 -> v2.0.0-rc6 2025-12-04T09:21:04.6850748Z * [new tag] v2.0.1 -> v2.0.1 2025-12-04T09:21:04.6851798Z * [new tag] v2.0.1-rc1 -> v2.0.1-rc1 2025-12-04T09:21:04.6852470Z * [new tag] v2.0.1-rc2 -> v2.0.1-rc2 2025-12-04T09:21:04.6853422Z * [new tag] v2.0.1-rc3 -> v2.0.1-rc3 2025-12-04T09:21:04.6854156Z * [new tag] v2.0.1-rc4 -> v2.0.1-rc4 2025-12-04T09:21:04.6855732Z * [new tag] v2.1.0 -> v2.1.0 2025-12-04T09:21:04.6856815Z * [new tag] v2.1.0-rc1 -> v2.1.0-rc1 2025-12-04T09:21:04.6857808Z * [new tag] v2.1.0-rc2 -> v2.1.0-rc2 2025-12-04T09:21:04.6858864Z * [new tag] v2.1.0-rc3 -> v2.1.0-rc3 2025-12-04T09:21:04.6859881Z * [new tag] v2.1.0-rc4 -> v2.1.0-rc4 2025-12-04T09:21:04.6860876Z * [new tag] v2.1.0-rc5 -> v2.1.0-rc5 2025-12-04T09:21:04.6861581Z * [new tag] v2.1.0-rc6 -> v2.1.0-rc6 2025-12-04T09:21:04.6862692Z * [new tag] v2.1.1 -> v2.1.1 2025-12-04T09:21:04.6863752Z * [new tag] v2.1.1-rc1 -> v2.1.1-rc1 2025-12-04T09:21:04.6864718Z * [new tag] v2.1.1-rc2 -> v2.1.1-rc2 2025-12-04T09:21:04.6865795Z * [new tag] v2.1.1-rc3 -> v2.1.1-rc3 2025-12-04T09:21:04.6866925Z * [new tag] v2.1.1-rc4 -> v2.1.1-rc4 2025-12-04T09:21:04.6868147Z * [new tag] v2.1.1-rc5 -> v2.1.1-rc5 2025-12-04T09:21:04.6868871Z * [new tag] v2.1.1-rc6 -> v2.1.1-rc6 2025-12-04T09:21:04.6869873Z * [new tag] v2.1.2 -> v2.1.2 2025-12-04T09:21:04.6870941Z * [new tag] v2.1.2-rc1 -> v2.1.2-rc1 2025-12-04T09:21:04.6871988Z * [new tag] v2.1.2-rc2 -> v2.1.2-rc2 2025-12-04T09:21:04.6872711Z * [new tag] v2.1.2-rc3 -> v2.1.2-rc3 2025-12-04T09:21:04.6873797Z * [new tag] v2.2.0 -> v2.2.0 2025-12-04T09:21:04.6874764Z * [new tag] v2.2.0-rc1 -> v2.2.0-rc1 2025-12-04T09:21:04.6876173Z * [new tag] v2.2.0-rc2 -> v2.2.0-rc2 2025-12-04T09:21:04.6877122Z * [new tag] v2.2.0-rc3 -> v2.2.0-rc3 2025-12-04T09:21:04.6878092Z * [new tag] v2.2.0-rc4 -> v2.2.0-rc4 2025-12-04T09:21:04.6879058Z * [new tag] v2.2.0-rc5 -> v2.2.0-rc5 2025-12-04T09:21:04.6880122Z * [new tag] v2.2.0-rc6 -> v2.2.0-rc6 2025-12-04T09:21:04.6880827Z * [new tag] v2.2.0-rc7 -> v2.2.0-rc7 2025-12-04T09:21:04.6881574Z * [new tag] v2.2.0-rc8 -> v2.2.0-rc8 2025-12-04T09:21:04.6882688Z * [new tag] v2.2.1 -> v2.2.1 2025-12-04T09:21:04.6883695Z * [new tag] v2.2.1-rc1 -> v2.2.1-rc1 2025-12-04T09:21:04.6884383Z * [new tag] v2.2.1-rc2 -> v2.2.1-rc2 2025-12-04T09:21:04.6885150Z * [new tag] v2.2.1-rc3 -> v2.2.1-rc3 2025-12-04T09:21:04.6885883Z * [new tag] v2.2.2 -> v2.2.2 2025-12-04T09:21:04.6887071Z * [new tag] v2.2.2-rc1 -> v2.2.2-rc1 2025-12-04T09:21:04.6887874Z * [new tag] v2.2.2-rc2 -> v2.2.2-rc2 2025-12-04T09:21:04.6888620Z * [new tag] v2.2.2-rc3 -> v2.2.2-rc3 2025-12-04T09:21:04.6889782Z * [new tag] v2.3.0 -> v2.3.0 2025-12-04T09:21:04.6890745Z * [new tag] v2.3.0-rc1 -> v2.3.0-rc1 2025-12-04T09:21:04.6891797Z * [new tag] v2.3.0-rc10 -> v2.3.0-rc10 2025-12-04T09:21:04.6892898Z * [new tag] v2.3.0-rc11 -> v2.3.0-rc11 2025-12-04T09:21:04.6893619Z * [new tag] v2.3.0-rc12 -> v2.3.0-rc12 2025-12-04T09:21:04.6894610Z * [new tag] v2.3.0-rc2 -> v2.3.0-rc2 2025-12-04T09:21:04.6895605Z * [new tag] v2.3.0-rc3 -> v2.3.0-rc3 2025-12-04T09:21:04.6896619Z * [new tag] v2.3.0-rc4 -> v2.3.0-rc4 2025-12-04T09:21:04.6897581Z * [new tag] v2.3.0-rc5 -> v2.3.0-rc5 2025-12-04T09:21:04.6898325Z * [new tag] v2.3.0-rc6 -> v2.3.0-rc6 2025-12-04T09:21:04.6904364Z * [new tag] v2.3.0-rc7 -> v2.3.0-rc7 2025-12-04T09:21:04.6904621Z * [new tag] v2.3.0-rc8 -> v2.3.0-rc8 2025-12-04T09:21:04.6904810Z * [new tag] v2.3.0-rc9 -> v2.3.0-rc9 2025-12-04T09:21:04.6904971Z * [new tag] v2.3.1 -> v2.3.1 2025-12-04T09:21:04.6905155Z * [new tag] v2.3.1-rc1 -> v2.3.1-rc1 2025-12-04T09:21:04.6905605Z * [new tag] v2.3.1-rc2 -> v2.3.1-rc2 2025-12-04T09:21:04.6906845Z * [new tag] v2.3.1-rc3 -> v2.3.1-rc3 2025-12-04T09:21:04.6907958Z * [new tag] v2.4.0 -> v2.4.0 2025-12-04T09:21:04.6908960Z * [new tag] v2.4.0-rc1 -> v2.4.0-rc1 2025-12-04T09:21:04.6909968Z * [new tag] v2.4.0-rc2 -> v2.4.0-rc2 2025-12-04T09:21:04.6910948Z * [new tag] v2.4.0-rc3 -> v2.4.0-rc3 2025-12-04T09:21:04.6911931Z * [new tag] v2.4.0-rc4 -> v2.4.0-rc4 2025-12-04T09:21:04.6912994Z * [new tag] v2.4.0-rc5 -> v2.4.0-rc5 2025-12-04T09:21:04.6913993Z * [new tag] v2.4.0-rc6 -> v2.4.0-rc6 2025-12-04T09:21:04.6915108Z * [new tag] v2.4.0-rc7 -> v2.4.0-rc7 2025-12-04T09:21:04.6916065Z * [new tag] v2.4.0-rc8 -> v2.4.0-rc8 2025-12-04T09:21:04.6917118Z * [new tag] v2.4.0-rc9 -> v2.4.0-rc9 2025-12-04T09:21:04.6917853Z * [new tag] v2.4.1 -> v2.4.1 2025-12-04T09:21:04.6919124Z * [new tag] v2.4.1-rc1 -> v2.4.1-rc1 2025-12-04T09:21:04.6920107Z * [new tag] v2.4.1-rc2 -> v2.4.1-rc2 2025-12-04T09:21:04.6921260Z * [new tag] v2.4.1-rc3 -> v2.4.1-rc3 2025-12-04T09:21:04.6922241Z * [new tag] v2.5.0 -> v2.5.0 2025-12-04T09:21:04.6923174Z * [new tag] v2.5.0-rc1 -> v2.5.0-rc1 2025-12-04T09:21:04.6923925Z * [new tag] v2.5.0-rc10 -> v2.5.0-rc10 2025-12-04T09:21:04.6924944Z * [new tag] v2.5.0-rc2 -> v2.5.0-rc2 2025-12-04T09:21:04.6925899Z * [new tag] v2.5.0-rc3 -> v2.5.0-rc3 2025-12-04T09:21:04.6926876Z * [new tag] v2.5.0-rc4 -> v2.5.0-rc4 2025-12-04T09:21:04.6927872Z * [new tag] v2.5.0-rc5 -> v2.5.0-rc5 2025-12-04T09:21:04.6928945Z * [new tag] v2.5.0-rc6 -> v2.5.0-rc6 2025-12-04T09:21:04.6929922Z * [new tag] v2.5.0-rc7 -> v2.5.0-rc7 2025-12-04T09:21:04.6930911Z * [new tag] v2.5.0-rc8 -> v2.5.0-rc8 2025-12-04T09:21:04.6931973Z * [new tag] v2.5.0-rc9 -> v2.5.0-rc9 2025-12-04T09:21:04.6932601Z * [new tag] v2.5.1 -> v2.5.1 2025-12-04T09:21:04.6933370Z * [new tag] v2.5.1-rc1 -> v2.5.1-rc1 2025-12-04T09:21:04.6934109Z * [new tag] v2.6.0 -> v2.6.0 2025-12-04T09:21:04.6935238Z * [new tag] v2.6.0-rc1 -> v2.6.0-rc1 2025-12-04T09:21:04.6936319Z * [new tag] v2.6.0-rc2 -> v2.6.0-rc2 2025-12-04T09:21:04.6937313Z * [new tag] v2.6.0-rc3 -> v2.6.0-rc3 2025-12-04T09:21:04.6938318Z * [new tag] v2.6.0-rc4 -> v2.6.0-rc4 2025-12-04T09:21:04.6939953Z * [new tag] v2.6.0-rc5 -> v2.6.0-rc5 2025-12-04T09:21:04.6941081Z * [new tag] v2.6.0-rc6 -> v2.6.0-rc6 2025-12-04T09:21:04.6942108Z * [new tag] v2.6.0-rc7 -> v2.6.0-rc7 2025-12-04T09:21:04.6943251Z * [new tag] v2.6.0-rc8 -> v2.6.0-rc8 2025-12-04T09:21:04.6944268Z * [new tag] v2.6.0-rc9 -> v2.6.0-rc9 2025-12-04T09:21:04.6945514Z * [new tag] v2.7.0 -> v2.7.0 2025-12-04T09:21:04.6946490Z * [new tag] v2.7.0-rc1 -> v2.7.0-rc1 2025-12-04T09:21:04.6947790Z * [new tag] v2.7.0-rc10 -> v2.7.0-rc10 2025-12-04T09:21:04.6948837Z * [new tag] v2.7.0-rc2 -> v2.7.0-rc2 2025-12-04T09:21:04.6949925Z * [new tag] v2.7.0-rc3 -> v2.7.0-rc3 2025-12-04T09:21:04.6950983Z * [new tag] v2.7.0-rc4 -> v2.7.0-rc4 2025-12-04T09:21:04.6951960Z * [new tag] v2.7.0-rc5 -> v2.7.0-rc5 2025-12-04T09:21:04.6952936Z * [new tag] v2.7.0-rc6 -> v2.7.0-rc6 2025-12-04T09:21:04.6954098Z * [new tag] v2.7.0-rc7 -> v2.7.0-rc7 2025-12-04T09:21:04.6955158Z * [new tag] v2.7.0-rc8 -> v2.7.0-rc8 2025-12-04T09:21:04.6956265Z * [new tag] v2.7.0-rc9 -> v2.7.0-rc9 2025-12-04T09:21:04.6957039Z * [new tag] v2.7.1 -> v2.7.1 2025-12-04T09:21:04.6958158Z * [new tag] v2.7.1-rc1 -> v2.7.1-rc1 2025-12-04T09:21:04.6959356Z * [new tag] v2.7.1-rc2 -> v2.7.1-rc2 2025-12-04T09:21:04.6960459Z * [new tag] v2.7.1-rc3 -> v2.7.1-rc3 2025-12-04T09:21:04.6966252Z * [new tag] v2.7.1-rc4 -> v2.7.1-rc4 2025-12-04T09:21:04.6966437Z * [new tag] v2.7.1-rc5 -> v2.7.1-rc5 2025-12-04T09:21:04.6966691Z * [new tag] v2.8.0 -> v2.8.0 2025-12-04T09:21:04.6966894Z * [new tag] v2.8.0-rc1 -> v2.8.0-rc1 2025-12-04T09:21:04.6967063Z * [new tag] v2.8.0-rc2 -> v2.8.0-rc2 2025-12-04T09:21:04.6967508Z * [new tag] v2.8.0-rc3 -> v2.8.0-rc3 2025-12-04T09:21:04.6968592Z * [new tag] v2.8.0-rc4 -> v2.8.0-rc4 2025-12-04T09:21:04.6969659Z * [new tag] v2.8.0-rc5 -> v2.8.0-rc5 2025-12-04T09:21:04.6970736Z * [new tag] v2.8.0-rc6 -> v2.8.0-rc6 2025-12-04T09:21:04.6971786Z * [new tag] v2.8.0-rc7 -> v2.8.0-rc7 2025-12-04T09:21:04.6972724Z * [new tag] v2.8.0-rc8 -> v2.8.0-rc8 2025-12-04T09:21:04.6973954Z * [new tag] v2.9.0 -> v2.9.0 2025-12-04T09:21:04.6975005Z * [new tag] v2.9.0-rc1 -> v2.9.0-rc1 2025-12-04T09:21:04.6976089Z * [new tag] v2.9.0-rc10 -> v2.9.0-rc10 2025-12-04T09:21:04.6977117Z * [new tag] v2.9.0-rc11 -> v2.9.0-rc11 2025-12-04T09:21:04.6978341Z * [new tag] v2.9.0-rc2 -> v2.9.0-rc2 2025-12-04T09:21:04.6979397Z * [new tag] v2.9.0-rc3 -> v2.9.0-rc3 2025-12-04T09:21:04.6980438Z * [new tag] v2.9.0-rc4 -> v2.9.0-rc4 2025-12-04T09:21:04.6981481Z * [new tag] v2.9.0-rc5 -> v2.9.0-rc5 2025-12-04T09:21:04.6982769Z * [new tag] v2.9.0-rc6 -> v2.9.0-rc6 2025-12-04T09:21:04.6983779Z * [new tag] v2.9.0-rc7 -> v2.9.0-rc7 2025-12-04T09:21:04.6985039Z * [new tag] v2.9.0-rc8 -> v2.9.0-rc8 2025-12-04T09:21:04.6985791Z * [new tag] v2.9.0-rc9 -> v2.9.0-rc9 2025-12-04T09:21:04.6986651Z * [new tag] v2.9.1 -> v2.9.1 2025-12-04T09:21:04.6988057Z * [new tag] v2.9.1-rc1 -> v2.9.1-rc1 2025-12-04T09:21:04.6989278Z * [new tag] v2.9.1-rc2 -> v2.9.1-rc2 2025-12-04T09:21:04.6990746Z * [new tag] viable/strict/1759343184 -> viable/strict/1759343184 2025-12-04T09:21:04.6991669Z * [new tag] viable/strict/1759346540 -> viable/strict/1759346540 2025-12-04T09:21:04.6992611Z * [new tag] viable/strict/1759348181 -> viable/strict/1759348181 2025-12-04T09:21:04.6993648Z * [new tag] viable/strict/1759350324 -> viable/strict/1759350324 2025-12-04T09:21:04.6994596Z * [new tag] viable/strict/1759351793 -> viable/strict/1759351793 2025-12-04T09:21:04.6995556Z * [new tag] viable/strict/1759353844 -> viable/strict/1759353844 2025-12-04T09:21:04.6996522Z * [new tag] viable/strict/1759355374 -> viable/strict/1759355374 2025-12-04T09:21:04.6997450Z * [new tag] viable/strict/1759357472 -> viable/strict/1759357472 2025-12-04T09:21:04.6998500Z * [new tag] viable/strict/1759361002 -> viable/strict/1759361002 2025-12-04T09:21:04.6999797Z * [new tag] viable/strict/1759362585 -> viable/strict/1759362585 2025-12-04T09:21:04.7002123Z * [new tag] viable/strict/1759365359 -> viable/strict/1759365359 2025-12-04T09:21:04.7003113Z * [new tag] viable/strict/1759370089 -> viable/strict/1759370089 2025-12-04T09:21:04.7004272Z * [new tag] viable/strict/1759377554 -> viable/strict/1759377554 2025-12-04T09:21:04.7005451Z * [new tag] viable/strict/1759379133 -> viable/strict/1759379133 2025-12-04T09:21:04.7006251Z * [new tag] viable/strict/1759389871 -> viable/strict/1759389871 2025-12-04T09:21:04.7007302Z * [new tag] viable/strict/1759393562 -> viable/strict/1759393562 2025-12-04T09:21:04.7008259Z * [new tag] viable/strict/1759395076 -> viable/strict/1759395076 2025-12-04T09:21:04.7009345Z * [new tag] viable/strict/1759398579 -> viable/strict/1759398579 2025-12-04T09:21:04.7010271Z * [new tag] viable/strict/1759404142 -> viable/strict/1759404142 2025-12-04T09:21:04.7011323Z * [new tag] viable/strict/1759405773 -> viable/strict/1759405773 2025-12-04T09:21:04.7012303Z * [new tag] viable/strict/1759408041 -> viable/strict/1759408041 2025-12-04T09:21:04.7013258Z * [new tag] viable/strict/1759411593 -> viable/strict/1759411593 2025-12-04T09:21:04.7014214Z * [new tag] viable/strict/1759427395 -> viable/strict/1759427395 2025-12-04T09:21:04.7015452Z * [new tag] viable/strict/1759434582 -> viable/strict/1759434582 2025-12-04T09:21:04.7016268Z * [new tag] viable/strict/1759436720 -> viable/strict/1759436720 2025-12-04T09:21:04.7017413Z * [new tag] viable/strict/1759440219 -> viable/strict/1759440219 2025-12-04T09:21:04.7018267Z * [new tag] viable/strict/1759441948 -> viable/strict/1759441948 2025-12-04T09:21:04.7019376Z * [new tag] viable/strict/1759443860 -> viable/strict/1759443860 2025-12-04T09:21:04.7020163Z * [new tag] viable/strict/1759445377 -> viable/strict/1759445377 2025-12-04T09:21:04.7021356Z * [new tag] viable/strict/1759447415 -> viable/strict/1759447415 2025-12-04T09:21:04.7022268Z * [new tag] viable/strict/1759451750 -> viable/strict/1759451750 2025-12-04T09:21:04.7023337Z * [new tag] viable/strict/1759453910 -> viable/strict/1759453910 2025-12-04T09:21:04.7029339Z * [new tag] viable/strict/1759456483 -> viable/strict/1759456483 2025-12-04T09:21:04.7030609Z * [new tag] viable/strict/1759459279 -> viable/strict/1759459279 2025-12-04T09:21:04.7031681Z * [new tag] viable/strict/1759460742 -> viable/strict/1759460742 2025-12-04T09:21:04.7032733Z * [new tag] viable/strict/1759462025 -> viable/strict/1759462025 2025-12-04T09:21:04.7033789Z * [new tag] viable/strict/1759469086 -> viable/strict/1759469086 2025-12-04T09:21:04.7034734Z * [new tag] viable/strict/1759470581 -> viable/strict/1759470581 2025-12-04T09:21:04.7035905Z * [new tag] viable/strict/1759472786 -> viable/strict/1759472786 2025-12-04T09:21:04.7036714Z * [new tag] viable/strict/1759476294 -> viable/strict/1759476294 2025-12-04T09:21:04.7037876Z * [new tag] viable/strict/1759479963 -> viable/strict/1759479963 2025-12-04T09:21:04.7038786Z * [new tag] viable/strict/1759492177 -> viable/strict/1759492177 2025-12-04T09:21:04.7039810Z * [new tag] viable/strict/1759519278 -> viable/strict/1759519278 2025-12-04T09:21:04.7040721Z * [new tag] viable/strict/1759524580 -> viable/strict/1759524580 2025-12-04T09:21:04.7041836Z * [new tag] viable/strict/1759528193 -> viable/strict/1759528193 2025-12-04T09:21:04.7042860Z * [new tag] viable/strict/1759533797 -> viable/strict/1759533797 2025-12-04T09:21:04.7043877Z * [new tag] viable/strict/1759542780 -> viable/strict/1759542780 2025-12-04T09:21:04.7044869Z * [new tag] viable/strict/1759549779 -> viable/strict/1759549779 2025-12-04T09:21:04.7045864Z * [new tag] viable/strict/1759555455 -> viable/strict/1759555455 2025-12-04T09:21:04.7046871Z * [new tag] viable/strict/1759559176 -> viable/strict/1759559176 2025-12-04T09:21:04.7047913Z * [new tag] viable/strict/1759560629 -> viable/strict/1759560629 2025-12-04T09:21:04.7048723Z * [new tag] viable/strict/1759569848 -> viable/strict/1759569848 2025-12-04T09:21:04.7049951Z * [new tag] viable/strict/1759571382 -> viable/strict/1759571382 2025-12-04T09:21:04.7050866Z * [new tag] viable/strict/1759573474 -> viable/strict/1759573474 2025-12-04T09:21:04.7051866Z * [new tag] viable/strict/1759618187 -> viable/strict/1759618187 2025-12-04T09:21:04.7052859Z * [new tag] viable/strict/1759626742 -> viable/strict/1759626742 2025-12-04T09:21:04.7053959Z * [new tag] viable/strict/1759632427 -> viable/strict/1759632427 2025-12-04T09:21:04.7054907Z * [new tag] viable/strict/1759634971 -> viable/strict/1759634971 2025-12-04T09:21:04.7055886Z * [new tag] viable/strict/1759661382 -> viable/strict/1759661382 2025-12-04T09:21:04.7056952Z * [new tag] viable/strict/1759663294 -> viable/strict/1759663294 2025-12-04T09:21:04.7057707Z * [new tag] viable/strict/1759708178 -> viable/strict/1759708178 2025-12-04T09:21:04.7059222Z * [new tag] viable/strict/1759715695 -> viable/strict/1759715695 2025-12-04T09:21:04.7060079Z * [new tag] viable/strict/1759728293 -> viable/strict/1759728293 2025-12-04T09:21:04.7061176Z * [new tag] viable/strict/1759735513 -> viable/strict/1759735513 2025-12-04T09:21:04.7062053Z * [new tag] viable/strict/1759739177 -> viable/strict/1759739177 2025-12-04T09:21:04.7063111Z * [new tag] viable/strict/1759758635 -> viable/strict/1759758635 2025-12-04T09:21:04.7064062Z * [new tag] viable/strict/1759765784 -> viable/strict/1759765784 2025-12-04T09:21:04.7065063Z * [new tag] viable/strict/1759767948 -> viable/strict/1759767948 2025-12-04T09:21:04.7066050Z * [new tag] viable/strict/1759771461 -> viable/strict/1759771461 2025-12-04T09:21:04.7066900Z * [new tag] viable/strict/1759776706 -> viable/strict/1759776706 2025-12-04T09:21:04.7068330Z * [new tag] viable/strict/1759782317 -> viable/strict/1759782317 2025-12-04T09:21:04.7069403Z * [new tag] viable/strict/1759783777 -> viable/strict/1759783777 2025-12-04T09:21:04.7070450Z * [new tag] viable/strict/1759785815 -> viable/strict/1759785815 2025-12-04T09:21:04.7071495Z * [new tag] viable/strict/1759789459 -> viable/strict/1759789459 2025-12-04T09:21:04.7072928Z * [new tag] viable/strict/1759790974 -> viable/strict/1759790974 2025-12-04T09:21:04.7073788Z * [new tag] viable/strict/1759794583 -> viable/strict/1759794583 2025-12-04T09:21:04.7074845Z * [new tag] viable/strict/1759797408 -> viable/strict/1759797408 2025-12-04T09:21:04.7075922Z * [new tag] viable/strict/1759799518 -> viable/strict/1759799518 2025-12-04T09:21:04.7076918Z * [new tag] viable/strict/1759804909 -> viable/strict/1759804909 2025-12-04T09:21:04.7077922Z * [new tag] viable/strict/1759807643 -> viable/strict/1759807643 2025-12-04T09:21:04.7079063Z * [new tag] viable/strict/1759809089 -> viable/strict/1759809089 2025-12-04T09:21:04.7080102Z * [new tag] viable/strict/1759811145 -> viable/strict/1759811145 2025-12-04T09:21:04.7080979Z * [new tag] viable/strict/1759812581 -> viable/strict/1759812581 2025-12-04T09:21:04.7081989Z * [new tag] viable/strict/1759814683 -> viable/strict/1759814683 2025-12-04T09:21:04.7082981Z * [new tag] viable/strict/1759821889 -> viable/strict/1759821889 2025-12-04T09:21:04.7083994Z * [new tag] viable/strict/1759823376 -> viable/strict/1759823376 2025-12-04T09:21:04.7085002Z * [new tag] viable/strict/1759827107 -> viable/strict/1759827107 2025-12-04T09:21:04.7085855Z * [new tag] viable/strict/1759830577 -> viable/strict/1759830577 2025-12-04T09:21:04.7091729Z * [new tag] viable/strict/1759832720 -> viable/strict/1759832720 2025-12-04T09:21:04.7093345Z * [new tag] viable/strict/1759842063 -> viable/strict/1759842063 2025-12-04T09:21:04.7094339Z * [new tag] viable/strict/1759847121 -> viable/strict/1759847121 2025-12-04T09:21:04.7095723Z * [new tag] viable/strict/1759850721 -> viable/strict/1759850721 2025-12-04T09:21:04.7096669Z * [new tag] viable/strict/1759857870 -> viable/strict/1759857870 2025-12-04T09:21:04.7097869Z * [new tag] viable/strict/1759863143 -> viable/strict/1759863143 2025-12-04T09:21:04.7098958Z * [new tag] viable/strict/1759875874 -> viable/strict/1759875874 2025-12-04T09:21:04.7099739Z * [new tag] viable/strict/1759877385 -> viable/strict/1759877385 2025-12-04T09:21:04.7101359Z * [new tag] viable/strict/1759883801 -> viable/strict/1759883801 2025-12-04T09:21:04.7102817Z * [new tag] viable/strict/1759885922 -> viable/strict/1759885922 2025-12-04T09:21:04.7103651Z * [new tag] viable/strict/1759888488 -> viable/strict/1759888488 2025-12-04T09:21:04.7145743Z * [new tag] viable/strict/1759895471 -> viable/strict/1759895471 2025-12-04T09:21:04.7146218Z * [new tag] viable/strict/1759904803 -> viable/strict/1759904803 2025-12-04T09:21:04.7146649Z * [new tag] viable/strict/1759908300 -> viable/strict/1759908300 2025-12-04T09:21:04.7146924Z * [new tag] viable/strict/1759915520 -> viable/strict/1759915520 2025-12-04T09:21:04.7147174Z * [new tag] viable/strict/1759916978 -> viable/strict/1759916978 2025-12-04T09:21:04.7147401Z * [new tag] viable/strict/1759930024 -> viable/strict/1759930024 2025-12-04T09:21:04.7147791Z * [new tag] viable/strict/1759948122 -> viable/strict/1759948122 2025-12-04T09:21:04.7148097Z * [new tag] viable/strict/1759952983 -> viable/strict/1759952983 2025-12-04T09:21:04.7148444Z * [new tag] viable/strict/1759955121 -> viable/strict/1759955121 2025-12-04T09:21:04.7148667Z * [new tag] viable/strict/1759962298 -> viable/strict/1759962298 2025-12-04T09:21:04.7148960Z * [new tag] viable/strict/1759965837 -> viable/strict/1759965837 2025-12-04T09:21:04.7149183Z * [new tag] viable/strict/1759970213 -> viable/strict/1759970213 2025-12-04T09:21:04.7154222Z * [new tag] viable/strict/1759974894 -> viable/strict/1759974894 2025-12-04T09:21:04.7154469Z * [new tag] viable/strict/1759977763 -> viable/strict/1759977763 2025-12-04T09:21:04.7154692Z * [new tag] viable/strict/1759979241 -> viable/strict/1759979241 2025-12-04T09:21:04.7154913Z * [new tag] viable/strict/1759985417 -> viable/strict/1759985417 2025-12-04T09:21:04.7155160Z * [new tag] viable/strict/1759987490 -> viable/strict/1759987490 2025-12-04T09:21:04.7155384Z * [new tag] viable/strict/1759996180 -> viable/strict/1759996180 2025-12-04T09:21:04.7155673Z * [new tag] viable/strict/1760065682 -> viable/strict/1760065682 2025-12-04T09:21:04.7155905Z * [new tag] viable/strict/1760066894 -> viable/strict/1760066894 2025-12-04T09:21:04.7156125Z * [new tag] viable/strict/1760070345 -> viable/strict/1760070345 2025-12-04T09:21:04.7156361Z * [new tag] viable/strict/1760089782 -> viable/strict/1760089782 2025-12-04T09:21:04.7156643Z * [new tag] viable/strict/1760091921 -> viable/strict/1760091921 2025-12-04T09:21:04.7156878Z * [new tag] viable/strict/1760127924 -> viable/strict/1760127924 2025-12-04T09:21:04.7157100Z * [new tag] viable/strict/1760129489 -> viable/strict/1760129489 2025-12-04T09:21:04.7157336Z * [new tag] viable/strict/1760132980 -> viable/strict/1760132980 2025-12-04T09:21:04.7157611Z * [new tag] viable/strict/1760135060 -> viable/strict/1760135060 2025-12-04T09:21:04.7157833Z * [new tag] viable/strict/1760215782 -> viable/strict/1760215782 2025-12-04T09:21:04.7158053Z * [new tag] viable/strict/1760273849 -> viable/strict/1760273849 2025-12-04T09:21:04.7158283Z * [new tag] viable/strict/1760275517 -> viable/strict/1760275517 2025-12-04T09:21:04.7158693Z * [new tag] viable/strict/1760276979 -> viable/strict/1760276979 2025-12-04T09:21:04.7158926Z * [new tag] viable/strict/1760279007 -> viable/strict/1760279007 2025-12-04T09:21:04.7159150Z * [new tag] viable/strict/1760286328 -> viable/strict/1760286328 2025-12-04T09:21:04.7159371Z * [new tag] viable/strict/1760493304 -> viable/strict/1760493304 2025-12-04T09:21:04.7159730Z * [new tag] viable/strict/1760496298 -> viable/strict/1760496298 2025-12-04T09:21:04.7159959Z * [new tag] viable/strict/1760518396 -> viable/strict/1760518396 2025-12-04T09:21:04.7160195Z * [new tag] viable/strict/1760534864 -> viable/strict/1760534864 2025-12-04T09:21:04.7160416Z * [new tag] viable/strict/1760549062 -> viable/strict/1760549062 2025-12-04T09:21:04.7160696Z * [new tag] viable/strict/1760552799 -> viable/strict/1760552799 2025-12-04T09:21:04.7160942Z * [new tag] viable/strict/1760554355 -> viable/strict/1760554355 2025-12-04T09:21:04.7161275Z * [new tag] viable/strict/1760556275 -> viable/strict/1760556275 2025-12-04T09:21:04.7161494Z * [new tag] viable/strict/1760564979 -> viable/strict/1760564979 2025-12-04T09:21:04.7161804Z * [new tag] viable/strict/1760567049 -> viable/strict/1760567049 2025-12-04T09:21:04.7162024Z * [new tag] viable/strict/1760568585 -> viable/strict/1760568585 2025-12-04T09:21:04.7162251Z * [new tag] viable/strict/1760570630 -> viable/strict/1760570630 2025-12-04T09:21:04.7162468Z * [new tag] viable/strict/1760572180 -> viable/strict/1760572180 2025-12-04T09:21:04.7162732Z * [new tag] viable/strict/1760575094 -> viable/strict/1760575094 2025-12-04T09:21:04.7162958Z * [new tag] viable/strict/1760579709 -> viable/strict/1760579709 2025-12-04T09:21:04.7163178Z * [new tag] viable/strict/1760582614 -> viable/strict/1760582614 2025-12-04T09:21:04.7163402Z * [new tag] viable/strict/1760586815 -> viable/strict/1760586815 2025-12-04T09:21:04.7163671Z * [new tag] viable/strict/1760588829 -> viable/strict/1760588829 2025-12-04T09:21:04.7163889Z * [new tag] viable/strict/1760590200 -> viable/strict/1760590200 2025-12-04T09:21:04.7164119Z * [new tag] viable/strict/1760592311 -> viable/strict/1760592311 2025-12-04T09:21:04.7164333Z * [new tag] viable/strict/1760619733 -> viable/strict/1760619733 2025-12-04T09:21:04.7164647Z * [new tag] viable/strict/1760628335 -> viable/strict/1760628335 2025-12-04T09:21:04.7164866Z * [new tag] viable/strict/1760635490 -> viable/strict/1760635490 2025-12-04T09:21:04.7165082Z * [new tag] viable/strict/1760640743 -> viable/strict/1760640743 2025-12-04T09:21:04.7165313Z * [new tag] viable/strict/1760642528 -> viable/strict/1760642528 2025-12-04T09:21:04.7165802Z * [new tag] viable/strict/1760646330 -> viable/strict/1760646330 2025-12-04T09:21:04.7166769Z * [new tag] viable/strict/1760666101 -> viable/strict/1760666101 2025-12-04T09:21:04.7167871Z * [new tag] viable/strict/1760668990 -> viable/strict/1760668990 2025-12-04T09:21:04.7168796Z * [new tag] viable/strict/1760670600 -> viable/strict/1760670600 2025-12-04T09:21:04.7169815Z * [new tag] viable/strict/1760671704 -> viable/strict/1760671704 2025-12-04T09:21:04.7170770Z * [new tag] viable/strict/1760673121 -> viable/strict/1760673121 2025-12-04T09:21:04.7171694Z * [new tag] viable/strict/1760675352 -> viable/strict/1760675352 2025-12-04T09:21:04.7172715Z * [new tag] viable/strict/1760696731 -> viable/strict/1760696731 2025-12-04T09:21:04.7175261Z * [new tag] viable/strict/1760723515 -> viable/strict/1760723515 2025-12-04T09:21:04.7176459Z * [new tag] viable/strict/1760727234 -> viable/strict/1760727234 2025-12-04T09:21:04.7177168Z * [new tag] viable/strict/1760730578 -> viable/strict/1760730578 2025-12-04T09:21:04.7178190Z * [new tag] viable/strict/1760732726 -> viable/strict/1760732726 2025-12-04T09:21:04.7179356Z * [new tag] viable/strict/1760734180 -> viable/strict/1760734180 2025-12-04T09:21:04.7180129Z * [new tag] viable/strict/1760736251 -> viable/strict/1760736251 2025-12-04T09:21:04.7181195Z * [new tag] viable/strict/1760737772 -> viable/strict/1760737772 2025-12-04T09:21:04.7182168Z * [new tag] viable/strict/1760758005 -> viable/strict/1760758005 2025-12-04T09:21:04.7183254Z * [new tag] viable/strict/1760761532 -> viable/strict/1760761532 2025-12-04T09:21:04.7184086Z * [new tag] viable/strict/1760802581 -> viable/strict/1760802581 2025-12-04T09:21:04.7185090Z * [new tag] viable/strict/1760827772 -> viable/strict/1760827772 2025-12-04T09:21:04.7186106Z * [new tag] viable/strict/1760834524 -> viable/strict/1760834524 2025-12-04T09:21:04.7187453Z * [new tag] viable/strict/1760845009 -> viable/strict/1760845009 2025-12-04T09:21:04.7188516Z * [new tag] viable/strict/1760876836 -> viable/strict/1760876836 2025-12-04T09:21:04.7189510Z * [new tag] viable/strict/1760880329 -> viable/strict/1760880329 2025-12-04T09:21:04.7190515Z * [new tag] viable/strict/1760888987 -> viable/strict/1760888987 2025-12-04T09:21:04.7191520Z * [new tag] viable/strict/1760912664 -> viable/strict/1760912664 2025-12-04T09:21:04.7192416Z * [new tag] viable/strict/1760925321 -> viable/strict/1760925321 2025-12-04T09:21:04.7193496Z * [new tag] viable/strict/1760931488 -> viable/strict/1760931488 2025-12-04T09:21:04.7194528Z * [new tag] viable/strict/1760932693 -> viable/strict/1760932693 2025-12-04T09:21:04.7195576Z * [new tag] viable/strict/1761004184 -> viable/strict/1761004184 2025-12-04T09:21:04.7196648Z * [new tag] viable/strict/1761014748 -> viable/strict/1761014748 2025-12-04T09:21:04.7197482Z * [new tag] viable/strict/1761017491 -> viable/strict/1761017491 2025-12-04T09:21:04.7198530Z * [new tag] viable/strict/1761018806 -> viable/strict/1761018806 2025-12-04T09:21:04.7199866Z * [new tag] viable/strict/1761020754 -> viable/strict/1761020754 2025-12-04T09:21:04.7201296Z * [new tag] viable/strict/1761024303 -> viable/strict/1761024303 2025-12-04T09:21:04.7205188Z * [new tag] viable/strict/1761029582 -> viable/strict/1761029582 2025-12-04T09:21:04.7206249Z * [new tag] viable/strict/1761031535 -> viable/strict/1761031535 2025-12-04T09:21:04.7207290Z * [new tag] viable/strict/1761035196 -> viable/strict/1761035196 2025-12-04T09:21:04.7208406Z * [new tag] viable/strict/1761045825 -> viable/strict/1761045825 2025-12-04T09:21:04.7209514Z * [new tag] viable/strict/1761054796 -> viable/strict/1761054796 2025-12-04T09:21:04.7210548Z * [new tag] viable/strict/1761060314 -> viable/strict/1761060314 2025-12-04T09:21:04.7216198Z * [new tag] viable/strict/1761071198 -> viable/strict/1761071198 2025-12-04T09:21:04.7216519Z * [new tag] viable/strict/1761074628 -> viable/strict/1761074628 2025-12-04T09:21:04.7216768Z * [new tag] viable/strict/1761078351 -> viable/strict/1761078351 2025-12-04T09:21:04.7217373Z * [new tag] viable/strict/1761079822 -> viable/strict/1761079822 2025-12-04T09:21:04.7217770Z * [new tag] viable/strict/1761081873 -> viable/strict/1761081873 2025-12-04T09:21:04.7218010Z * [new tag] viable/strict/1761083392 -> viable/strict/1761083392 2025-12-04T09:21:04.7218611Z * [new tag] viable/strict/1761085465 -> viable/strict/1761085465 2025-12-04T09:21:04.7219724Z * [new tag] viable/strict/1761089099 -> viable/strict/1761089099 2025-12-04T09:21:04.7220721Z * [new tag] viable/strict/1761095535 -> viable/strict/1761095535 2025-12-04T09:21:04.7221970Z * [new tag] viable/strict/1761098119 -> viable/strict/1761098119 2025-12-04T09:21:04.7223618Z * [new tag] viable/strict/1761101330 -> viable/strict/1761101330 2025-12-04T09:21:04.7224643Z * [new tag] viable/strict/1761114425 -> viable/strict/1761114425 2025-12-04T09:21:04.7225775Z * [new tag] viable/strict/1761116036 -> viable/strict/1761116036 2025-12-04T09:21:04.7226656Z * [new tag] viable/strict/1761119379 -> viable/strict/1761119379 2025-12-04T09:21:04.7228006Z * [new tag] viable/strict/1761121601 -> viable/strict/1761121601 2025-12-04T09:21:04.7228925Z * [new tag] viable/strict/1761123234 -> viable/strict/1761123234 2025-12-04T09:21:04.7229916Z * [new tag] viable/strict/1761126621 -> viable/strict/1761126621 2025-12-04T09:21:04.7230888Z * [new tag] viable/strict/1761132259 -> viable/strict/1761132259 2025-12-04T09:21:04.7232005Z * [new tag] viable/strict/1761146746 -> viable/strict/1761146746 2025-12-04T09:21:04.7233073Z * [new tag] viable/strict/1761164752 -> viable/strict/1761164752 2025-12-04T09:21:04.7233997Z * [new tag] viable/strict/1761166198 -> viable/strict/1761166198 2025-12-04T09:21:04.7235099Z * [new tag] viable/strict/1761175424 -> viable/strict/1761175424 2025-12-04T09:21:04.7236038Z * [new tag] viable/strict/1761176983 -> viable/strict/1761176983 2025-12-04T09:21:04.7237231Z * [new tag] viable/strict/1761179891 -> viable/strict/1761179891 2025-12-04T09:21:04.7238455Z * [new tag] viable/strict/1761181930 -> viable/strict/1761181930 2025-12-04T09:21:04.7239524Z * [new tag] viable/strict/1761184516 -> viable/strict/1761184516 2025-12-04T09:21:04.7240599Z * [new tag] viable/strict/1761190179 -> viable/strict/1761190179 2025-12-04T09:21:04.7241480Z * [new tag] viable/strict/1761193558 -> viable/strict/1761193558 2025-12-04T09:21:04.7242510Z * [new tag] viable/strict/1761207990 -> viable/strict/1761207990 2025-12-04T09:21:04.7243427Z * [new tag] viable/strict/1761229539 -> viable/strict/1761229539 2025-12-04T09:21:04.7244696Z * [new tag] viable/strict/1761244031 -> viable/strict/1761244031 2025-12-04T09:21:04.7245608Z * [new tag] viable/strict/1761248986 -> viable/strict/1761248986 2025-12-04T09:21:04.7246647Z * [new tag] viable/strict/1761259791 -> viable/strict/1761259791 2025-12-04T09:21:04.7247602Z * [new tag] viable/strict/1761266139 -> viable/strict/1761266139 2025-12-04T09:21:04.7248713Z * [new tag] viable/strict/1761268316 -> viable/strict/1761268316 2025-12-04T09:21:04.7249661Z * [new tag] viable/strict/1761273805 -> viable/strict/1761273805 2025-12-04T09:21:04.7250648Z * [new tag] viable/strict/1761275261 -> viable/strict/1761275261 2025-12-04T09:21:04.7251665Z * [new tag] viable/strict/1761277913 -> viable/strict/1761277913 2025-12-04T09:21:04.7252743Z * [new tag] viable/strict/1761290701 -> viable/strict/1761290701 2025-12-04T09:21:04.7253794Z * [new tag] viable/strict/1761294396 -> viable/strict/1761294396 2025-12-04T09:21:04.7254724Z * [new tag] viable/strict/1761303047 -> viable/strict/1761303047 2025-12-04T09:21:04.7255715Z * [new tag] viable/strict/1761335388 -> viable/strict/1761335388 2025-12-04T09:21:04.7256842Z * [new tag] viable/strict/1761337551 -> viable/strict/1761337551 2025-12-04T09:21:04.7258046Z * [new tag] viable/strict/1761339007 -> viable/strict/1761339007 2025-12-04T09:21:04.7258735Z * [new tag] viable/strict/1761341050 -> viable/strict/1761341050 2025-12-04T09:21:04.7259783Z * [new tag] viable/strict/1761346188 -> viable/strict/1761346188 2025-12-04T09:21:04.7260876Z * [new tag] viable/strict/1761349792 -> viable/strict/1761349792 2025-12-04T09:21:04.7261946Z * [new tag] viable/strict/1761352620 -> viable/strict/1761352620 2025-12-04T09:21:04.7262840Z * [new tag] viable/strict/1761354730 -> viable/strict/1761354730 2025-12-04T09:21:04.7263912Z * [new tag] viable/strict/1761357298 -> viable/strict/1761357298 2025-12-04T09:21:04.7264942Z * [new tag] viable/strict/1761360201 -> viable/strict/1761360201 2025-12-04T09:21:04.7265933Z * [new tag] viable/strict/1761361753 -> viable/strict/1761361753 2025-12-04T09:21:04.7267107Z * [new tag] viable/strict/1761364351 -> viable/strict/1761364351 2025-12-04T09:21:04.7268361Z * [new tag] viable/strict/1761366338 -> viable/strict/1761366338 2025-12-04T09:21:04.7269507Z * [new tag] viable/strict/1761367802 -> viable/strict/1761367802 2025-12-04T09:21:04.7270648Z * [new tag] viable/strict/1761369889 -> viable/strict/1761369889 2025-12-04T09:21:04.7271712Z * [new tag] viable/strict/1761371385 -> viable/strict/1761371385 2025-12-04T09:21:04.7272797Z * [new tag] viable/strict/1761373581 -> viable/strict/1761373581 2025-12-04T09:21:04.7273978Z * [new tag] viable/strict/1761375054 -> viable/strict/1761375054 2025-12-04T09:21:04.7280163Z * [new tag] viable/strict/1761421785 -> viable/strict/1761421785 2025-12-04T09:21:04.7281291Z * [new tag] viable/strict/1761434614 -> viable/strict/1761434614 2025-12-04T09:21:04.7282670Z * [new tag] viable/strict/1761439254 -> viable/strict/1761439254 2025-12-04T09:21:04.7283730Z * [new tag] viable/strict/1761454187 -> viable/strict/1761454187 2025-12-04T09:21:04.7284862Z * [new tag] viable/strict/1761459991 -> viable/strict/1761459991 2025-12-04T09:21:04.7286050Z * [new tag] viable/strict/1761470668 -> viable/strict/1761470668 2025-12-04T09:21:04.7287567Z * [new tag] viable/strict/1761472188 -> viable/strict/1761472188 2025-12-04T09:21:04.7288617Z * [new tag] viable/strict/1761503178 -> viable/strict/1761503178 2025-12-04T09:21:04.7289621Z * [new tag] viable/strict/1761517492 -> viable/strict/1761517492 2025-12-04T09:21:04.7290752Z * [new tag] viable/strict/1761518981 -> viable/strict/1761518981 2025-12-04T09:21:04.7292092Z * [new tag] viable/strict/1761533609 -> viable/strict/1761533609 2025-12-04T09:21:04.7292908Z * [new tag] viable/strict/1761546438 -> viable/strict/1761546438 2025-12-04T09:21:04.7294070Z * [new tag] viable/strict/1761548133 -> viable/strict/1761548133 2025-12-04T09:21:04.7295370Z * [new tag] viable/strict/1761555186 -> viable/strict/1761555186 2025-12-04T09:21:04.7296461Z * [new tag] viable/strict/1761557178 -> viable/strict/1761557178 2025-12-04T09:21:04.7297457Z * [new tag] viable/strict/1761560772 -> viable/strict/1761560772 2025-12-04T09:21:04.7298515Z * [new tag] viable/strict/1761562266 -> viable/strict/1761562266 2025-12-04T09:21:04.7299486Z * [new tag] viable/strict/1761564260 -> viable/strict/1761564260 2025-12-04T09:21:04.7300517Z * [new tag] viable/strict/1761568072 -> viable/strict/1761568072 2025-12-04T09:21:04.7302424Z * [new tag] viable/strict/1761571683 -> viable/strict/1761571683 2025-12-04T09:21:04.7303343Z * [new tag] viable/strict/1761580199 -> viable/strict/1761580199 2025-12-04T09:21:04.7304202Z * [new tag] viable/strict/1761587383 -> viable/strict/1761587383 2025-12-04T09:21:04.7305395Z * [new tag] viable/strict/1761591165 -> viable/strict/1761591165 2025-12-04T09:21:04.7306410Z * [new tag] viable/strict/1761594575 -> viable/strict/1761594575 2025-12-04T09:21:04.7307650Z * [new tag] viable/strict/1761596710 -> viable/strict/1761596710 2025-12-04T09:21:04.7308652Z * [new tag] viable/strict/1761598189 -> viable/strict/1761598189 2025-12-04T09:21:04.7309717Z * [new tag] viable/strict/1761600254 -> viable/strict/1761600254 2025-12-04T09:21:04.7310778Z * [new tag] viable/strict/1761603879 -> viable/strict/1761603879 2025-12-04T09:21:04.7311828Z * [new tag] viable/strict/1761605429 -> viable/strict/1761605429 2025-12-04T09:21:04.7313064Z * [new tag] viable/strict/1761607468 -> viable/strict/1761607468 2025-12-04T09:21:04.7314118Z * [new tag] viable/strict/1761608983 -> viable/strict/1761608983 2025-12-04T09:21:04.7315164Z * [new tag] viable/strict/1761611846 -> viable/strict/1761611846 2025-12-04T09:21:04.7316212Z * [new tag] viable/strict/1761613922 -> viable/strict/1761613922 2025-12-04T09:21:04.7317057Z * [new tag] viable/strict/1761616504 -> viable/strict/1761616504 2025-12-04T09:21:04.7317913Z * [new tag] viable/strict/1761619599 -> viable/strict/1761619599 2025-12-04T09:21:04.7319121Z * [new tag] viable/strict/1761686693 -> viable/strict/1761686693 2025-12-04T09:21:04.7320110Z * [new tag] viable/strict/1761688179 -> viable/strict/1761688179 2025-12-04T09:21:04.7321073Z * [new tag] viable/strict/1761691973 -> viable/strict/1761691973 2025-12-04T09:21:04.7322247Z * [new tag] viable/strict/1761693884 -> viable/strict/1761693884 2025-12-04T09:21:04.7323359Z * [new tag] viable/strict/1761695389 -> viable/strict/1761695389 2025-12-04T09:21:04.7324353Z * [new tag] viable/strict/1761698408 -> viable/strict/1761698408 2025-12-04T09:21:04.7325335Z * [new tag] viable/strict/1761702931 -> viable/strict/1761702931 2025-12-04T09:21:04.7326347Z * [new tag] viable/strict/1761706307 -> viable/strict/1761706307 2025-12-04T09:21:04.7327401Z * [new tag] viable/strict/1761709065 -> viable/strict/1761709065 2025-12-04T09:21:04.7328489Z * [new tag] viable/strict/1761710285 -> viable/strict/1761710285 2025-12-04T09:21:04.7329551Z * [new tag] viable/strict/1761711983 -> viable/strict/1761711983 2025-12-04T09:21:04.7330642Z * [new tag] viable/strict/1761713514 -> viable/strict/1761713514 2025-12-04T09:21:04.7331792Z * [new tag] viable/strict/1761715523 -> viable/strict/1761715523 2025-12-04T09:21:04.7332834Z * [new tag] viable/strict/1761727973 -> viable/strict/1761727973 2025-12-04T09:21:04.7333916Z * [new tag] viable/strict/1761751558 -> viable/strict/1761751558 2025-12-04T09:21:04.7335000Z * [new tag] viable/strict/1761755187 -> viable/strict/1761755187 2025-12-04T09:21:04.7336170Z * [new tag] viable/strict/1761756826 -> viable/strict/1761756826 2025-12-04T09:21:04.7342351Z * [new tag] viable/strict/1761769551 -> viable/strict/1761769551 2025-12-04T09:21:04.7343550Z * [new tag] viable/strict/1761771032 -> viable/strict/1761771032 2025-12-04T09:21:04.7344414Z * [new tag] viable/strict/1761773101 -> viable/strict/1761773101 2025-12-04T09:21:04.7345600Z * [new tag] viable/strict/1761781792 -> viable/strict/1761781792 2025-12-04T09:21:04.7346929Z * [new tag] viable/strict/1761784788 -> viable/strict/1761784788 2025-12-04T09:21:04.7348042Z * [new tag] viable/strict/1761786740 -> viable/strict/1761786740 2025-12-04T09:21:04.7349101Z * [new tag] viable/strict/1761789332 -> viable/strict/1761789332 2025-12-04T09:21:04.7350717Z * [new tag] viable/strict/1761792569 -> viable/strict/1761792569 2025-12-04T09:21:04.7351787Z * [new tag] viable/strict/1761795289 -> viable/strict/1761795289 2025-12-04T09:21:04.7352957Z * [new tag] viable/strict/1761798345 -> viable/strict/1761798345 2025-12-04T09:21:04.7354023Z * [new tag] viable/strict/1761799827 -> viable/strict/1761799827 2025-12-04T09:21:04.7355174Z * [new tag] viable/strict/1761805604 -> viable/strict/1761805604 2025-12-04T09:21:04.7356236Z * [new tag] viable/strict/1761807202 -> viable/strict/1761807202 2025-12-04T09:21:04.7357282Z * [new tag] viable/strict/1761809094 -> viable/strict/1761809094 2025-12-04T09:21:04.7358432Z * [new tag] viable/strict/1761810576 -> viable/strict/1761810576 2025-12-04T09:21:04.7359488Z * [new tag] viable/strict/1761812771 -> viable/strict/1761812771 2025-12-04T09:21:04.7360528Z * [new tag] viable/strict/1761814363 -> viable/strict/1761814363 2025-12-04T09:21:04.7361576Z * [new tag] viable/strict/1761857410 -> viable/strict/1761857410 2025-12-04T09:21:04.7362591Z * [new tag] viable/strict/1761860985 -> viable/strict/1761860985 2025-12-04T09:21:04.7363788Z * [new tag] viable/strict/1761863094 -> viable/strict/1761863094 2025-12-04T09:21:04.7364687Z * [new tag] viable/strict/1761864590 -> viable/strict/1761864590 2025-12-04T09:21:04.7365750Z * [new tag] viable/strict/1761866675 -> viable/strict/1761866675 2025-12-04T09:21:04.7367674Z * [new tag] viable/strict/1761868178 -> viable/strict/1761868178 2025-12-04T09:21:04.7368753Z * [new tag] viable/strict/1761871111 -> viable/strict/1761871111 2025-12-04T09:21:04.7369806Z * [new tag] viable/strict/1761873126 -> viable/strict/1761873126 2025-12-04T09:21:04.7370868Z * [new tag] viable/strict/1761875714 -> viable/strict/1761875714 2025-12-04T09:21:04.7371914Z * [new tag] viable/strict/1761878924 -> viable/strict/1761878924 2025-12-04T09:21:04.7372973Z * [new tag] viable/strict/1761881727 -> viable/strict/1761881727 2025-12-04T09:21:04.7374053Z * [new tag] viable/strict/1761882959 -> viable/strict/1761882959 2025-12-04T09:21:04.7375094Z * [new tag] viable/strict/1761886268 -> viable/strict/1761886268 2025-12-04T09:21:04.7376117Z * [new tag] viable/strict/1761893641 -> viable/strict/1761893641 2025-12-04T09:21:04.7377191Z * [new tag] viable/strict/1761931517 -> viable/strict/1761931517 2025-12-04T09:21:04.7378244Z * [new tag] viable/strict/1761933080 -> viable/strict/1761933080 2025-12-04T09:21:04.7379252Z * [new tag] viable/strict/1761935217 -> viable/strict/1761935217 2025-12-04T09:21:04.7380317Z * [new tag] viable/strict/1761938533 -> viable/strict/1761938533 2025-12-04T09:21:04.7381436Z * [new tag] viable/strict/1761940184 -> viable/strict/1761940184 2025-12-04T09:21:04.7382479Z * [new tag] viable/strict/1761942338 -> viable/strict/1761942338 2025-12-04T09:21:04.7383486Z * [new tag] viable/strict/1761946100 -> viable/strict/1761946100 2025-12-04T09:21:04.7384551Z * [new tag] viable/strict/1761947374 -> viable/strict/1761947374 2025-12-04T09:21:04.7385614Z * [new tag] viable/strict/1761950978 -> viable/strict/1761950978 2025-12-04T09:21:04.7386813Z * [new tag] viable/strict/1761957727 -> viable/strict/1761957727 2025-12-04T09:21:04.7388233Z * [new tag] viable/strict/1761959532 -> viable/strict/1761959532 2025-12-04T09:21:04.7389338Z * [new tag] viable/strict/1761965366 -> viable/strict/1761965366 2025-12-04T09:21:04.7390797Z * [new tag] viable/strict/1761968066 -> viable/strict/1761968066 2025-12-04T09:21:04.7391912Z * [new tag] viable/strict/1761969322 -> viable/strict/1761969322 2025-12-04T09:21:04.7392990Z * [new tag] viable/strict/1761974723 -> viable/strict/1761974723 2025-12-04T09:21:04.7394144Z * [new tag] viable/strict/1761981837 -> viable/strict/1761981837 2025-12-04T09:21:04.7395224Z * [new tag] viable/strict/1761985546 -> viable/strict/1761985546 2025-12-04T09:21:04.7396310Z * [new tag] viable/strict/1761987030 -> viable/strict/1761987030 2025-12-04T09:21:04.7397526Z * [new tag] viable/strict/1762003554 -> viable/strict/1762003554 2025-12-04T09:21:04.7398602Z * [new tag] viable/strict/1762021560 -> viable/strict/1762021560 2025-12-04T09:21:04.7404357Z * [new tag] viable/strict/1762032190 -> viable/strict/1762032190 2025-12-04T09:21:04.7404647Z * [new tag] viable/strict/1762040981 -> viable/strict/1762040981 2025-12-04T09:21:04.7405388Z * [new tag] viable/strict/1762048525 -> viable/strict/1762048525 2025-12-04T09:21:04.7406704Z * [new tag] viable/strict/1762104223 -> viable/strict/1762104223 2025-12-04T09:21:04.7407775Z * [new tag] viable/strict/1762105778 -> viable/strict/1762105778 2025-12-04T09:21:04.7408844Z * [new tag] viable/strict/1762115109 -> viable/strict/1762115109 2025-12-04T09:21:04.7409868Z * [new tag] viable/strict/1762125840 -> viable/strict/1762125840 2025-12-04T09:21:04.7410787Z * [new tag] viable/strict/1762127377 -> viable/strict/1762127377 2025-12-04T09:21:04.7412250Z * [new tag] viable/strict/1762134925 -> viable/strict/1762134925 2025-12-04T09:21:04.7413257Z * [new tag] viable/strict/1762138338 -> viable/strict/1762138338 2025-12-04T09:21:04.7414423Z * [new tag] viable/strict/1762148993 -> viable/strict/1762148993 2025-12-04T09:21:04.7415491Z * [new tag] viable/strict/1762152871 -> viable/strict/1762152871 2025-12-04T09:21:04.7416528Z * [new tag] viable/strict/1762156183 -> viable/strict/1762156183 2025-12-04T09:21:04.7417542Z * [new tag] viable/strict/1762163457 -> viable/strict/1762163457 2025-12-04T09:21:04.7418588Z * [new tag] viable/strict/1762165569 -> viable/strict/1762165569 2025-12-04T09:21:04.7419605Z * [new tag] viable/strict/1762169035 -> viable/strict/1762169035 2025-12-04T09:21:04.7420654Z * [new tag] viable/strict/1762174936 -> viable/strict/1762174936 2025-12-04T09:21:04.7421734Z * [new tag] viable/strict/1762194412 -> viable/strict/1762194412 2025-12-04T09:21:04.7422769Z * [new tag] viable/strict/1762195876 -> viable/strict/1762195876 2025-12-04T09:21:04.7423795Z * [new tag] viable/strict/1762197788 -> viable/strict/1762197788 2025-12-04T09:21:04.7424856Z * [new tag] viable/strict/1762199389 -> viable/strict/1762199389 2025-12-04T09:21:04.7426160Z * [new tag] viable/strict/1762206585 -> viable/strict/1762206585 2025-12-04T09:21:04.7427876Z * [new tag] viable/strict/1762210184 -> viable/strict/1762210184 2025-12-04T09:21:04.7428681Z * [new tag] viable/strict/1762218736 -> viable/strict/1762218736 2025-12-04T09:21:04.7429839Z * [new tag] viable/strict/1762224529 -> viable/strict/1762224529 2025-12-04T09:21:04.7431107Z * [new tag] viable/strict/1762227253 -> viable/strict/1762227253 2025-12-04T09:21:04.7431890Z * [new tag] viable/strict/1762228515 -> viable/strict/1762228515 2025-12-04T09:21:04.7433122Z * [new tag] viable/strict/1762230349 -> viable/strict/1762230349 2025-12-04T09:21:04.7434167Z * [new tag] viable/strict/1762231859 -> viable/strict/1762231859 2025-12-04T09:21:04.7435251Z * [new tag] viable/strict/1762233925 -> viable/strict/1762233925 2025-12-04T09:21:04.7436587Z * [new tag] viable/strict/1762237630 -> viable/strict/1762237630 2025-12-04T09:21:04.7437471Z * [new tag] viable/strict/1762253522 -> viable/strict/1762253522 2025-12-04T09:21:04.7438722Z * [new tag] viable/strict/1762278588 -> viable/strict/1762278588 2025-12-04T09:21:04.7439895Z * [new tag] viable/strict/1762284203 -> viable/strict/1762284203 2025-12-04T09:21:04.7440949Z * [new tag] viable/strict/1762289446 -> viable/strict/1762289446 2025-12-04T09:21:04.7442486Z * [new tag] viable/strict/1762291515 -> viable/strict/1762291515 2025-12-04T09:21:04.7443554Z * [new tag] viable/strict/1762295100 -> viable/strict/1762295100 2025-12-04T09:21:04.7444398Z * [new tag] viable/strict/1762296590 -> viable/strict/1762296590 2025-12-04T09:21:04.7445376Z * [new tag] viable/strict/1762300179 -> viable/strict/1762300179 2025-12-04T09:21:04.7446202Z * [new tag] viable/strict/1762303207 -> viable/strict/1762303207 2025-12-04T09:21:04.7447342Z * [new tag] viable/strict/1762386584 -> viable/strict/1762386584 2025-12-04T09:21:04.7448381Z * [new tag] viable/strict/1762391537 -> viable/strict/1762391537 2025-12-04T09:21:04.7449235Z * [new tag] viable/strict/1762394119 -> viable/strict/1762394119 2025-12-04T09:21:04.7450735Z * [new tag] viable/strict/1762397437 -> viable/strict/1762397437 2025-12-04T09:21:04.7451766Z * [new tag] viable/strict/1762400256 -> viable/strict/1762400256 2025-12-04T09:21:04.7452830Z * [new tag] viable/strict/1762401469 -> viable/strict/1762401469 2025-12-04T09:21:04.7453967Z * [new tag] viable/strict/1762408195 -> viable/strict/1762408195 2025-12-04T09:21:04.7455058Z * [new tag] viable/strict/1762410411 -> viable/strict/1762410411 2025-12-04T09:21:04.7456138Z * [new tag] viable/strict/1762417613 -> viable/strict/1762417613 2025-12-04T09:21:04.7457200Z * [new tag] viable/strict/1762419198 -> viable/strict/1762419198 2025-12-04T09:21:04.7458260Z * [new tag] viable/strict/1762422656 -> viable/strict/1762422656 2025-12-04T09:21:04.7459730Z * [new tag] viable/strict/1762424746 -> viable/strict/1762424746 2025-12-04T09:21:04.7460847Z * [new tag] viable/strict/1762446386 -> viable/strict/1762446386 2025-12-04T09:21:04.7466255Z * [new tag] viable/strict/1762449912 -> viable/strict/1762449912 2025-12-04T09:21:04.7466666Z * [new tag] viable/strict/1762457031 -> viable/strict/1762457031 2025-12-04T09:21:04.7466952Z * [new tag] viable/strict/1762462441 -> viable/strict/1762462441 2025-12-04T09:21:04.7467178Z * [new tag] viable/strict/1762467909 -> viable/strict/1762467909 2025-12-04T09:21:04.7467419Z * [new tag] viable/strict/1762471493 -> viable/strict/1762471493 2025-12-04T09:21:04.7468249Z * [new tag] viable/strict/1762475990 -> viable/strict/1762475990 2025-12-04T09:21:04.7469537Z * [new tag] viable/strict/1762477933 -> viable/strict/1762477933 2025-12-04T09:21:04.7470625Z * [new tag] viable/strict/1762491053 -> viable/strict/1762491053 2025-12-04T09:21:04.7471787Z * [new tag] viable/strict/1762493118 -> viable/strict/1762493118 2025-12-04T09:21:04.7472766Z * [new tag] viable/strict/1762498442 -> viable/strict/1762498442 2025-12-04T09:21:04.7473845Z * [new tag] viable/strict/1762501778 -> viable/strict/1762501778 2025-12-04T09:21:04.7474946Z * [new tag] viable/strict/1762504001 -> viable/strict/1762504001 2025-12-04T09:21:04.7476171Z * [new tag] viable/strict/1762505583 -> viable/strict/1762505583 2025-12-04T09:21:04.7477343Z * [new tag] viable/strict/1762507523 -> viable/strict/1762507523 2025-12-04T09:21:04.7478564Z * [new tag] viable/strict/1762511140 -> viable/strict/1762511140 2025-12-04T09:21:04.7479773Z * [new tag] viable/strict/1762512632 -> viable/strict/1762512632 2025-12-04T09:21:04.7480858Z * [new tag] viable/strict/1762520467 -> viable/strict/1762520467 2025-12-04T09:21:04.7481931Z * [new tag] viable/strict/1762522016 -> viable/strict/1762522016 2025-12-04T09:21:04.7482971Z * [new tag] viable/strict/1762530591 -> viable/strict/1762530591 2025-12-04T09:21:04.7484017Z * [new tag] viable/strict/1762543405 -> viable/strict/1762543405 2025-12-04T09:21:04.7484869Z * [new tag] viable/strict/1762544998 -> viable/strict/1762544998 2025-12-04T09:21:04.7485967Z * [new tag] viable/strict/1762552182 -> viable/strict/1762552182 2025-12-04T09:21:04.7487029Z * [new tag] viable/strict/1762554297 -> viable/strict/1762554297 2025-12-04T09:21:04.7487885Z * [new tag] viable/strict/1762559381 -> viable/strict/1762559381 2025-12-04T09:21:04.7488994Z * [new tag] viable/strict/1762562222 -> viable/strict/1762562222 2025-12-04T09:21:04.7490087Z * [new tag] viable/strict/1762564319 -> viable/strict/1762564319 2025-12-04T09:21:04.7490917Z * [new tag] viable/strict/1762566904 -> viable/strict/1762566904 2025-12-04T09:21:04.7492013Z * [new tag] viable/strict/1762569781 -> viable/strict/1762569781 2025-12-04T09:21:04.7493032Z * [new tag] viable/strict/1762575940 -> viable/strict/1762575940 2025-12-04T09:21:04.7494135Z * [new tag] viable/strict/1762580974 -> viable/strict/1762580974 2025-12-04T09:21:04.7495255Z * [new tag] viable/strict/1762583185 -> viable/strict/1762583185 2025-12-04T09:21:04.7496266Z * [new tag] viable/strict/1762586647 -> viable/strict/1762586647 2025-12-04T09:21:04.7497357Z * [new tag] viable/strict/1762588183 -> viable/strict/1762588183 2025-12-04T09:21:04.7498508Z * [new tag] viable/strict/1762593886 -> viable/strict/1762593886 2025-12-04T09:21:04.7499570Z * [new tag] viable/strict/1762650743 -> viable/strict/1762650743 2025-12-04T09:21:04.7500718Z * [new tag] viable/strict/1762653328 -> viable/strict/1762653328 2025-12-04T09:21:04.7502598Z * [new tag] viable/strict/1762659342 -> viable/strict/1762659342 2025-12-04T09:21:04.7503716Z * [new tag] viable/strict/1762662360 -> viable/strict/1762662360 2025-12-04T09:21:04.7504790Z * [new tag] viable/strict/1762667377 -> viable/strict/1762667377 2025-12-04T09:21:04.7505881Z * [new tag] viable/strict/1762671090 -> viable/strict/1762671090 2025-12-04T09:21:04.7507054Z * [new tag] viable/strict/1762680284 -> viable/strict/1762680284 2025-12-04T09:21:04.7508277Z * [new tag] viable/strict/1762683900 -> viable/strict/1762683900 2025-12-04T09:21:04.7509422Z * [new tag] viable/strict/1762705541 -> viable/strict/1762705541 2025-12-04T09:21:04.7510484Z * [new tag] viable/strict/1762709004 -> viable/strict/1762709004 2025-12-04T09:21:04.7511696Z * [new tag] viable/strict/1762746004 -> viable/strict/1762746004 2025-12-04T09:21:04.7512824Z * [new tag] viable/strict/1762748799 -> viable/strict/1762748799 2025-12-04T09:21:04.7514360Z * [new tag] viable/strict/1762759504 -> viable/strict/1762759504 2025-12-04T09:21:04.7515569Z * [new tag] viable/strict/1762760973 -> viable/strict/1762760973 2025-12-04T09:21:04.7516650Z * [new tag] viable/strict/1762775374 -> viable/strict/1762775374 2025-12-04T09:21:04.7517767Z * [new tag] viable/strict/1762777661 -> viable/strict/1762777661 2025-12-04T09:21:04.7518996Z * [new tag] viable/strict/1762779774 -> viable/strict/1762779774 2025-12-04T09:21:04.7520230Z * [new tag] viable/strict/1762781259 -> viable/strict/1762781259 2025-12-04T09:21:04.7521350Z * [new tag] viable/strict/1762793628 -> viable/strict/1762793628 2025-12-04T09:21:04.7522467Z * [new tag] viable/strict/1762800711 -> viable/strict/1762800711 2025-12-04T09:21:04.7523641Z * [new tag] viable/strict/1762809894 -> viable/strict/1762809894 2025-12-04T09:21:04.7529749Z * [new tag] viable/strict/1762811384 -> viable/strict/1762811384 2025-12-04T09:21:04.7530974Z * [new tag] viable/strict/1762813841 -> viable/strict/1762813841 2025-12-04T09:21:04.7532086Z * [new tag] viable/strict/1762815047 -> viable/strict/1762815047 2025-12-04T09:21:04.7533377Z * [new tag] viable/strict/1762817094 -> viable/strict/1762817094 2025-12-04T09:21:04.7534445Z * [new tag] viable/strict/1762818582 -> viable/strict/1762818582 2025-12-04T09:21:04.7535547Z * [new tag] viable/strict/1762821623 -> viable/strict/1762821623 2025-12-04T09:21:04.7536539Z * [new tag] viable/strict/1762823531 -> viable/strict/1762823531 2025-12-04T09:21:04.7537771Z * [new tag] viable/strict/1762849583 -> viable/strict/1762849583 2025-12-04T09:21:04.7538907Z * [new tag] viable/strict/1762851200 -> viable/strict/1762851200 2025-12-04T09:21:04.7539935Z * [new tag] viable/strict/1762854603 -> viable/strict/1762854603 2025-12-04T09:21:04.7541164Z * [new tag] viable/strict/1762858276 -> viable/strict/1762858276 2025-12-04T09:21:04.7542256Z * [new tag] viable/strict/1762860891 -> viable/strict/1762860891 2025-12-04T09:21:04.7544004Z * [new tag] viable/strict/1762866174 -> viable/strict/1762866174 2025-12-04T09:21:04.7545096Z * [new tag] viable/strict/1762867653 -> viable/strict/1762867653 2025-12-04T09:21:04.7546086Z * [new tag] viable/strict/1762872669 -> viable/strict/1762872669 2025-12-04T09:21:04.7547276Z * [new tag] viable/strict/1762878380 -> viable/strict/1762878380 2025-12-04T09:21:04.7548544Z * [new tag] viable/strict/1762889003 -> viable/strict/1762889003 2025-12-04T09:21:04.7549688Z * [new tag] viable/strict/1762890589 -> viable/strict/1762890589 2025-12-04T09:21:04.7550775Z * [new tag] viable/strict/1762892743 -> viable/strict/1762892743 2025-12-04T09:21:04.7551862Z * [new tag] viable/strict/1762894271 -> viable/strict/1762894271 2025-12-04T09:21:04.7552754Z * [new tag] viable/strict/1762896287 -> viable/strict/1762896287 2025-12-04T09:21:04.7553933Z * [new tag] viable/strict/1762915871 -> viable/strict/1762915871 2025-12-04T09:21:04.7555128Z * [new tag] viable/strict/1762918569 -> viable/strict/1762918569 2025-12-04T09:21:04.7555987Z * [new tag] viable/strict/1762919776 -> viable/strict/1762919776 2025-12-04T09:21:04.7557124Z * [new tag] viable/strict/1762923072 -> viable/strict/1762923072 2025-12-04T09:21:04.7558296Z * [new tag] viable/strict/1762928826 -> viable/strict/1762928826 2025-12-04T09:21:04.7559627Z * [new tag] viable/strict/1762930451 -> viable/strict/1762930451 2025-12-04T09:21:04.7560682Z * [new tag] viable/strict/1762933780 -> viable/strict/1762933780 2025-12-04T09:21:04.7561747Z * [new tag] viable/strict/1762937638 -> viable/strict/1762937638 2025-12-04T09:21:04.7563275Z * [new tag] viable/strict/1762939545 -> viable/strict/1762939545 2025-12-04T09:21:04.7564359Z * [new tag] viable/strict/1762962692 -> viable/strict/1762962692 2025-12-04T09:21:04.7565405Z * [new tag] viable/strict/1762979143 -> viable/strict/1762979143 2025-12-04T09:21:04.7566483Z * [new tag] viable/strict/1762984188 -> viable/strict/1762984188 2025-12-04T09:21:04.7567349Z * [new tag] viable/strict/1762986306 -> viable/strict/1762986306 2025-12-04T09:21:04.7568476Z * [new tag] viable/strict/1762989903 -> viable/strict/1762989903 2025-12-04T09:21:04.7569523Z * [new tag] viable/strict/1762991377 -> viable/strict/1762991377 2025-12-04T09:21:04.7570634Z * [new tag] viable/strict/1762998921 -> viable/strict/1762998921 2025-12-04T09:21:04.7571802Z * [new tag] viable/strict/1763002287 -> viable/strict/1763002287 2025-12-04T09:21:04.7572896Z * [new tag] viable/strict/1763016840 -> viable/strict/1763016840 2025-12-04T09:21:04.7573954Z * [new tag] viable/strict/1763020180 -> viable/strict/1763020180 2025-12-04T09:21:04.7575105Z * [new tag] viable/strict/1763027421 -> viable/strict/1763027421 2025-12-04T09:21:04.7576192Z * [new tag] viable/strict/1763031120 -> viable/strict/1763031120 2025-12-04T09:21:04.7577365Z * [new tag] viable/strict/1763036861 -> viable/strict/1763036861 2025-12-04T09:21:04.7578466Z * [new tag] viable/strict/1763038993 -> viable/strict/1763038993 2025-12-04T09:21:04.7579631Z * [new tag] viable/strict/1763054703 -> viable/strict/1763054703 2025-12-04T09:21:04.7580496Z * [new tag] viable/strict/1763067061 -> viable/strict/1763067061 2025-12-04T09:21:04.7581613Z * [new tag] viable/strict/1763070847 -> viable/strict/1763070847 2025-12-04T09:21:04.7582710Z * [new tag] viable/strict/1763072706 -> viable/strict/1763072706 2025-12-04T09:21:04.7583851Z * [new tag] viable/strict/1763076302 -> viable/strict/1763076302 2025-12-04T09:21:04.7584955Z * [new tag] viable/strict/1763080816 -> viable/strict/1763080816 2025-12-04T09:21:04.7586105Z * [new tag] viable/strict/1763082732 -> viable/strict/1763082732 2025-12-04T09:21:04.7592438Z * [new tag] viable/strict/1763085329 -> viable/strict/1763085329 2025-12-04T09:21:04.7593624Z * [new tag] viable/strict/1763088623 -> viable/strict/1763088623 2025-12-04T09:21:04.7594842Z * [new tag] viable/strict/1763091402 -> viable/strict/1763091402 2025-12-04T09:21:04.7595979Z * [new tag] viable/strict/1763092602 -> viable/strict/1763092602 2025-12-04T09:21:04.7597557Z * [new tag] viable/strict/1763094355 -> viable/strict/1763094355 2025-12-04T09:21:04.7598685Z * [new tag] viable/strict/1763099390 -> viable/strict/1763099390 2025-12-04T09:21:04.7599869Z * [new tag] viable/strict/1763101608 -> viable/strict/1763101608 2025-12-04T09:21:04.7601855Z * [new tag] viable/strict/1763105102 -> viable/strict/1763105102 2025-12-04T09:21:04.7606193Z * [new tag] viable/strict/1763112347 -> viable/strict/1763112347 2025-12-04T09:21:04.7607367Z * [new tag] viable/strict/1763119471 -> viable/strict/1763119471 2025-12-04T09:21:04.7608348Z * [new tag] viable/strict/1763126835 -> viable/strict/1763126835 2025-12-04T09:21:04.7609169Z * [new tag] viable/strict/1763149779 -> viable/strict/1763149779 2025-12-04T09:21:04.7610350Z * [new tag] viable/strict/1763164178 -> viable/strict/1763164178 2025-12-04T09:21:04.7611451Z * [new tag] viable/strict/1763167104 -> viable/strict/1763167104 2025-12-04T09:21:04.7612505Z * [new tag] viable/strict/1763169132 -> viable/strict/1763169132 2025-12-04T09:21:04.7613712Z * [new tag] viable/strict/1763171708 -> viable/strict/1763171708 2025-12-04T09:21:04.7614812Z * [new tag] viable/strict/1763174759 -> viable/strict/1763174759 2025-12-04T09:21:04.7615937Z * [new tag] viable/strict/1763180744 -> viable/strict/1763180744 2025-12-04T09:21:04.7616965Z * [new tag] viable/strict/1763182227 -> viable/strict/1763182227 2025-12-04T09:21:04.7618000Z * [new tag] viable/strict/1763184309 -> viable/strict/1763184309 2025-12-04T09:21:04.7619682Z * [new tag] viable/strict/1763187991 -> viable/strict/1763187991 2025-12-04T09:21:04.7620790Z * [new tag] viable/strict/1763191445 -> viable/strict/1763191445 2025-12-04T09:21:04.7621950Z * [new tag] viable/strict/1763195152 -> viable/strict/1763195152 2025-12-04T09:21:04.7622985Z * [new tag] viable/strict/1763205769 -> viable/strict/1763205769 2025-12-04T09:21:04.7624060Z * [new tag] viable/strict/1763246990 -> viable/strict/1763246990 2025-12-04T09:21:04.7625299Z * [new tag] viable/strict/1763261578 -> viable/strict/1763261578 2025-12-04T09:21:04.7629412Z * [new tag] viable/strict/1763286573 -> viable/strict/1763286573 2025-12-04T09:21:04.7629640Z * [new tag] viable/strict/1763292167 -> viable/strict/1763292167 2025-12-04T09:21:04.7629884Z * [new tag] viable/strict/1763333386 -> viable/strict/1763333386 2025-12-04T09:21:04.7630109Z * [new tag] viable/strict/1763340082 -> viable/strict/1763340082 2025-12-04T09:21:04.7631698Z * [new tag] viable/strict/1763364324 -> viable/strict/1763364324 2025-12-04T09:21:04.7632832Z * [new tag] viable/strict/1763371569 -> viable/strict/1763371569 2025-12-04T09:21:04.7633872Z * [new tag] viable/strict/1763373067 -> viable/strict/1763373067 2025-12-04T09:21:04.7634941Z * [new tag] viable/strict/1763375157 -> viable/strict/1763375157 2025-12-04T09:21:04.7636018Z * [new tag] viable/strict/1763382462 -> viable/strict/1763382462 2025-12-04T09:21:04.7637209Z * [new tag] viable/strict/1763394661 -> viable/strict/1763394661 2025-12-04T09:21:04.7638494Z * [new tag] viable/strict/1763396797 -> viable/strict/1763396797 2025-12-04T09:21:04.7639758Z * [new tag] viable/strict/1763398542 -> viable/strict/1763398542 2025-12-04T09:21:04.7640823Z * [new tag] viable/strict/1763401807 -> viable/strict/1763401807 2025-12-04T09:21:04.7641697Z * [new tag] viable/strict/1763414698 -> viable/strict/1763414698 2025-12-04T09:21:04.7642912Z * [new tag] viable/strict/1763419807 -> viable/strict/1763419807 2025-12-04T09:21:04.7643972Z * [new tag] viable/strict/1763426369 -> viable/strict/1763426369 2025-12-04T09:21:04.7645076Z * [new tag] viable/strict/1763428331 -> viable/strict/1763428331 2025-12-04T09:21:04.7646180Z * [new tag] viable/strict/1763430922 -> viable/strict/1763430922 2025-12-04T09:21:04.7647020Z * [new tag] viable/strict/1763434184 -> viable/strict/1763434184 2025-12-04T09:21:04.7648133Z * [new tag] viable/strict/1763439973 -> viable/strict/1763439973 2025-12-04T09:21:04.7653880Z * [new tag] viable/strict/1763444995 -> viable/strict/1763444995 2025-12-04T09:21:04.7654176Z * [new tag] viable/strict/1763447206 -> viable/strict/1763447206 2025-12-04T09:21:04.7654441Z * [new tag] viable/strict/1763448826 -> viable/strict/1763448826 2025-12-04T09:21:04.7654915Z * [new tag] viable/strict/1763450717 -> viable/strict/1763450717 2025-12-04T09:21:04.7655160Z * [new tag] viable/strict/1763452183 -> viable/strict/1763452183 2025-12-04T09:21:04.7655650Z * [new tag] viable/strict/1763457945 -> viable/strict/1763457945 2025-12-04T09:21:04.7656780Z * [new tag] viable/strict/1763459439 -> viable/strict/1763459439 2025-12-04T09:21:04.7657684Z * [new tag] viable/strict/1763461556 -> viable/strict/1763461556 2025-12-04T09:21:04.7658806Z * [new tag] viable/strict/1763463103 -> viable/strict/1763463103 2025-12-04T09:21:04.7660065Z * [new tag] viable/strict/1763465100 -> viable/strict/1763465100 2025-12-04T09:21:04.7660975Z * [new tag] viable/strict/1763468866 -> viable/strict/1763468866 2025-12-04T09:21:04.7661910Z * [new tag] viable/strict/1763493823 -> viable/strict/1763493823 2025-12-04T09:21:04.7662860Z * [new tag] viable/strict/1763496249 -> viable/strict/1763496249 2025-12-04T09:21:04.7664013Z * [new tag] viable/strict/1763502620 -> viable/strict/1763502620 2025-12-04T09:21:04.7665189Z * [new tag] viable/strict/1763504715 -> viable/strict/1763504715 2025-12-04T09:21:04.7666306Z * [new tag] viable/strict/1763506208 -> viable/strict/1763506208 2025-12-04T09:21:04.7667627Z * [new tag] viable/strict/1763520590 -> viable/strict/1763520590 2025-12-04T09:21:04.7668780Z * [new tag] viable/strict/1763523357 -> viable/strict/1763523357 2025-12-04T09:21:04.7669966Z * [new tag] viable/strict/1763529922 -> viable/strict/1763529922 2025-12-04T09:21:04.7671152Z * [new tag] viable/strict/1763531408 -> viable/strict/1763531408 2025-12-04T09:21:04.7672225Z * [new tag] viable/strict/1763533622 -> viable/strict/1763533622 2025-12-04T09:21:04.7673352Z * [new tag] viable/strict/1763538576 -> viable/strict/1763538576 2025-12-04T09:21:04.7675012Z * [new tag] viable/strict/1763545823 -> viable/strict/1763545823 2025-12-04T09:21:04.7675881Z * [new tag] viable/strict/1763547951 -> viable/strict/1763547951 2025-12-04T09:21:04.7677101Z * [new tag] viable/strict/1763551477 -> viable/strict/1763551477 2025-12-04T09:21:04.7678190Z * [new tag] viable/strict/1763552982 -> viable/strict/1763552982 2025-12-04T09:21:04.7679403Z * [new tag] viable/strict/1763594698 -> viable/strict/1763594698 2025-12-04T09:21:04.7680535Z * [new tag] viable/strict/1763596178 -> viable/strict/1763596178 2025-12-04T09:21:04.7681629Z * [new tag] viable/strict/1763599155 -> viable/strict/1763599155 2025-12-04T09:21:04.7682667Z * [new tag] viable/strict/1763603717 -> viable/strict/1763603717 2025-12-04T09:21:04.7683756Z * [new tag] viable/strict/1763606923 -> viable/strict/1763606923 2025-12-04T09:21:04.7684824Z * [new tag] viable/strict/1763609715 -> viable/strict/1763609715 2025-12-04T09:21:04.7685912Z * [new tag] viable/strict/1763612757 -> viable/strict/1763612757 2025-12-04T09:21:04.7686937Z * [new tag] viable/strict/1763616325 -> viable/strict/1763616325 2025-12-04T09:21:04.7688025Z * [new tag] viable/strict/1763623509 -> viable/strict/1763623509 2025-12-04T09:21:04.7689243Z * [new tag] viable/strict/1763624984 -> viable/strict/1763624984 2025-12-04T09:21:04.7690410Z * [new tag] viable/strict/1763628796 -> viable/strict/1763628796 2025-12-04T09:21:04.7691287Z * [new tag] viable/strict/1763634343 -> viable/strict/1763634343 2025-12-04T09:21:04.7692348Z * [new tag] viable/strict/1763635867 -> viable/strict/1763635867 2025-12-04T09:21:04.7693598Z * [new tag] viable/strict/1763639382 -> viable/strict/1763639382 2025-12-04T09:21:04.7694652Z * [new tag] viable/strict/1763646626 -> viable/strict/1763646626 2025-12-04T09:21:04.7696003Z * [new tag] viable/strict/1763655997 -> viable/strict/1763655997 2025-12-04T09:21:04.7697076Z * [new tag] viable/strict/1763659444 -> viable/strict/1763659444 2025-12-04T09:21:04.7698122Z * [new tag] viable/strict/1763660992 -> viable/strict/1763660992 2025-12-04T09:21:04.7699156Z * [new tag] viable/strict/1763663201 -> viable/strict/1763663201 2025-12-04T09:21:04.7700241Z * [new tag] viable/strict/1763670362 -> viable/strict/1763670362 2025-12-04T09:21:04.7701832Z * [new tag] viable/strict/1763675378 -> viable/strict/1763675378 2025-12-04T09:21:04.7703057Z * [new tag] viable/strict/1763693343 -> viable/strict/1763693343 2025-12-04T09:21:04.7704179Z * [new tag] viable/strict/1763696088 -> viable/strict/1763696088 2025-12-04T09:21:04.7705393Z * [new tag] viable/strict/1763697343 -> viable/strict/1763697343 2025-12-04T09:21:04.7706485Z * [new tag] viable/strict/1763699165 -> viable/strict/1763699165 2025-12-04T09:21:04.7707783Z * [new tag] viable/strict/1763700660 -> viable/strict/1763700660 2025-12-04T09:21:04.7708891Z * [new tag] viable/strict/1763704209 -> viable/strict/1763704209 2025-12-04T09:21:04.7709989Z * [new tag] viable/strict/1763706411 -> viable/strict/1763706411 2025-12-04T09:21:04.7711069Z * [new tag] viable/strict/1763708082 -> viable/strict/1763708082 2025-12-04T09:21:04.7716189Z * [new tag] viable/strict/1763711381 -> viable/strict/1763711381 2025-12-04T09:21:04.7716464Z * [new tag] viable/strict/1763713593 -> viable/strict/1763713593 2025-12-04T09:21:04.7716693Z * [new tag] viable/strict/1763715201 -> viable/strict/1763715201 2025-12-04T09:21:04.7716936Z * [new tag] viable/strict/1763733017 -> viable/strict/1763733017 2025-12-04T09:21:04.7717159Z * [new tag] viable/strict/1763735108 -> viable/strict/1763735108 2025-12-04T09:21:04.7717672Z * [new tag] viable/strict/1763749579 -> viable/strict/1763749579 2025-12-04T09:21:04.7718757Z * [new tag] viable/strict/1763751113 -> viable/strict/1763751113 2025-12-04T09:21:04.7719897Z * [new tag] viable/strict/1763753035 -> viable/strict/1763753035 2025-12-04T09:21:04.7721039Z * [new tag] viable/strict/1763754578 -> viable/strict/1763754578 2025-12-04T09:21:04.7722101Z * [new tag] viable/strict/1763756748 -> viable/strict/1763756748 2025-12-04T09:21:04.7723186Z * [new tag] viable/strict/1763758205 -> viable/strict/1763758205 2025-12-04T09:21:04.7724038Z * [new tag] viable/strict/1763764050 -> viable/strict/1763764050 2025-12-04T09:21:04.7725160Z * [new tag] viable/strict/1763771887 -> viable/strict/1763771887 2025-12-04T09:21:04.7726551Z * [new tag] viable/strict/1763773920 -> viable/strict/1763773920 2025-12-04T09:21:04.7727553Z * [new tag] viable/strict/1763776501 -> viable/strict/1763776501 2025-12-04T09:21:04.7728615Z * [new tag] viable/strict/1763779437 -> viable/strict/1763779437 2025-12-04T09:21:04.7729920Z * [new tag] viable/strict/1763781038 -> viable/strict/1763781038 2025-12-04T09:21:04.7730861Z * [new tag] viable/strict/1763782245 -> viable/strict/1763782245 2025-12-04T09:21:04.7732042Z * [new tag] viable/strict/1763785568 -> viable/strict/1763785568 2025-12-04T09:21:04.7733066Z * [new tag] viable/strict/1763787006 -> viable/strict/1763787006 2025-12-04T09:21:04.7734215Z * [new tag] viable/strict/1763789103 -> viable/strict/1763789103 2025-12-04T09:21:04.7735375Z * [new tag] viable/strict/1763790578 -> viable/strict/1763790578 2025-12-04T09:21:04.7736747Z * [new tag] viable/strict/1763796275 -> viable/strict/1763796275 2025-12-04T09:21:04.7738093Z * [new tag] viable/strict/1763801465 -> viable/strict/1763801465 2025-12-04T09:21:04.7739135Z * [new tag] viable/strict/1763803522 -> viable/strict/1763803522 2025-12-04T09:21:04.7740184Z * [new tag] viable/strict/1763808581 -> viable/strict/1763808581 2025-12-04T09:21:04.7741251Z * [new tag] viable/strict/1763840977 -> viable/strict/1763840977 2025-12-04T09:21:04.7742269Z * [new tag] viable/strict/1763846659 -> viable/strict/1763846659 2025-12-04T09:21:04.7743298Z * [new tag] viable/strict/1763872065 -> viable/strict/1763872065 2025-12-04T09:21:04.7744410Z * [new tag] viable/strict/1763873648 -> viable/strict/1763873648 2025-12-04T09:21:04.7745499Z * [new tag] viable/strict/1763875506 -> viable/strict/1763875506 2025-12-04T09:21:04.7747112Z * [new tag] viable/strict/1763889904 -> viable/strict/1763889904 2025-12-04T09:21:04.7748288Z * [new tag] viable/strict/1763930999 -> viable/strict/1763930999 2025-12-04T09:21:04.7749410Z * [new tag] viable/strict/1763944964 -> viable/strict/1763944964 2025-12-04T09:21:04.7750275Z * [new tag] viable/strict/1763958474 -> viable/strict/1763958474 2025-12-04T09:21:04.7751383Z * [new tag] viable/strict/1763967263 -> viable/strict/1763967263 2025-12-04T09:21:04.7752528Z * [new tag] viable/strict/1763972803 -> viable/strict/1763972803 2025-12-04T09:21:04.7753559Z * [new tag] viable/strict/1763976376 -> viable/strict/1763976376 2025-12-04T09:21:04.7754657Z * [new tag] viable/strict/1763989404 -> viable/strict/1763989404 2025-12-04T09:21:04.7755695Z * [new tag] viable/strict/1763990887 -> viable/strict/1763990887 2025-12-04T09:21:04.7756816Z * [new tag] viable/strict/1764019919 -> viable/strict/1764019919 2025-12-04T09:21:04.7757939Z * [new tag] viable/strict/1764023134 -> viable/strict/1764023134 2025-12-04T09:21:04.7759113Z * [new tag] viable/strict/1764024593 -> viable/strict/1764024593 2025-12-04T09:21:04.7760115Z * [new tag] viable/strict/1764026706 -> viable/strict/1764026706 2025-12-04T09:21:04.7761441Z * [new tag] viable/strict/1764031139 -> viable/strict/1764031139 2025-12-04T09:21:04.7762540Z * [new tag] viable/strict/1764033131 -> viable/strict/1764033131 2025-12-04T09:21:04.7763364Z * [new tag] viable/strict/1764035725 -> viable/strict/1764035725 2025-12-04T09:21:04.7764305Z * [new tag] viable/strict/1764624265 -> viable/strict/1764624265 2025-12-04T09:21:04.7765152Z * [new tag] viable/strict/1764631514 -> viable/strict/1764631514 2025-12-04T09:21:04.7766002Z * [new tag] viable/strict/1764632987 -> viable/strict/1764632987 2025-12-04T09:21:04.7767056Z * [new tag] viable/strict/1764636063 -> viable/strict/1764636063 2025-12-04T09:21:04.7767880Z * [new tag] viable/strict/1764643975 -> viable/strict/1764643975 2025-12-04T09:21:04.7769134Z * [new tag] viable/strict/1764646859 -> viable/strict/1764646859 2025-12-04T09:21:04.7769804Z * [new tag] viable/strict/1764653120 -> viable/strict/1764653120 2025-12-04T09:21:04.7770501Z * [new tag] viable/strict/1764654632 -> viable/strict/1764654632 2025-12-04T09:21:04.7771370Z * [new tag] viable/strict/1764656821 -> viable/strict/1764656821 2025-12-04T09:21:04.7772335Z * [new tag] viable/strict/1764658557 -> viable/strict/1764658557 2025-12-04T09:21:04.7773113Z * [new tag] viable/strict/1764660333 -> viable/strict/1764660333 2025-12-04T09:21:04.7779537Z * [new tag] viable/strict/1764661812 -> viable/strict/1764661812 2025-12-04T09:21:04.7780229Z * [new tag] viable/strict/1764664023 -> viable/strict/1764664023 2025-12-04T09:21:04.7781147Z * [new tag] viable/strict/1764669150 -> viable/strict/1764669150 2025-12-04T09:21:04.7782028Z * [new tag] viable/strict/1764680709 -> viable/strict/1764680709 2025-12-04T09:21:04.7782948Z * [new tag] viable/strict/1764687619 -> viable/strict/1764687619 2025-12-04T09:21:04.7783873Z * [new tag] viable/strict/1764696355 -> viable/strict/1764696355 2025-12-04T09:21:04.7784780Z * [new tag] viable/strict/1764701767 -> viable/strict/1764701767 2025-12-04T09:21:04.7785676Z * [new tag] viable/strict/1764710768 -> viable/strict/1764710768 2025-12-04T09:21:04.7786714Z * [new tag] viable/strict/1764716202 -> viable/strict/1764716202 2025-12-04T09:21:04.7787666Z * [new tag] viable/strict/1764793566 -> viable/strict/1764793566 2025-12-04T09:21:04.7788583Z * [new tag] viable/strict/1764797093 -> viable/strict/1764797093 2025-12-04T09:21:04.7789475Z * [new tag] viable/strict/1764800729 -> viable/strict/1764800729 2025-12-04T09:21:04.7790732Z * [new tag] whc_flight_1 -> whc_flight_1 2025-12-04T09:21:04.7791774Z * [new tag] whc_flight_2 -> whc_flight_2 2025-12-04T09:21:04.7793069Z * [new tag] whc_flight_4 -> whc_flight_4 2025-12-04T09:21:04.8680813Z [command]/usr/bin/git rev-parse --verify --quiet ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32^{object} 2025-12-04T09:21:04.8708579Z ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:21:04.8720225Z ##[endgroup] 2025-12-04T09:21:04.8720586Z ##[group]Determining the checkout info 2025-12-04T09:21:04.8721203Z ##[endgroup] 2025-12-04T09:21:04.8726110Z [command]/usr/bin/git sparse-checkout disable 2025-12-04T09:21:04.8761132Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig 2025-12-04T09:21:04.8790161Z ##[group]Checking out the ref 2025-12-04T09:21:04.8793993Z [command]/usr/bin/git checkout --progress --force ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:21:05.9610513Z Updating files: 75% (15103/20121) 2025-12-04T09:21:05.9770492Z Updating files: 76% (15292/20121) 2025-12-04T09:21:05.9925042Z Updating files: 77% (15494/20121) 2025-12-04T09:21:06.0159744Z Updating files: 78% (15695/20121) 2025-12-04T09:21:06.0466874Z Updating files: 79% (15896/20121) 2025-12-04T09:21:06.0831937Z Updating files: 80% (16097/20121) 2025-12-04T09:21:06.1172028Z Updating files: 81% (16299/20121) 2025-12-04T09:21:06.1418207Z Updating files: 82% (16500/20121) 2025-12-04T09:21:06.1583686Z Updating files: 83% (16701/20121) 2025-12-04T09:21:06.1747262Z Updating files: 84% (16902/20121) 2025-12-04T09:21:06.1935827Z Updating files: 85% (17103/20121) 2025-12-04T09:21:06.2116281Z Updating files: 86% (17305/20121) 2025-12-04T09:21:06.2274691Z Updating files: 87% (17506/20121) 2025-12-04T09:21:06.2408144Z Updating files: 88% (17707/20121) 2025-12-04T09:21:06.2561287Z Updating files: 89% (17908/20121) 2025-12-04T09:21:06.2762100Z Updating files: 90% (18109/20121) 2025-12-04T09:21:06.2894495Z Updating files: 91% (18311/20121) 2025-12-04T09:21:06.3068846Z Updating files: 92% (18512/20121) 2025-12-04T09:21:06.3282349Z Updating files: 93% (18713/20121) 2025-12-04T09:21:06.3517247Z Updating files: 94% (18914/20121) 2025-12-04T09:21:06.3718822Z Updating files: 95% (19115/20121) 2025-12-04T09:21:06.3904020Z Updating files: 96% (19317/20121) 2025-12-04T09:21:06.4083604Z Updating files: 97% (19518/20121) 2025-12-04T09:21:06.4418182Z Updating files: 98% (19719/20121) 2025-12-04T09:21:06.4617698Z Updating files: 99% (19920/20121) 2025-12-04T09:21:06.4618112Z Updating files: 100% (20121/20121) 2025-12-04T09:21:06.4618476Z Updating files: 100% (20121/20121), done. 2025-12-04T09:21:06.4938509Z Note: switching to 'ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32'. 2025-12-04T09:21:06.4938946Z 2025-12-04T09:21:06.4939217Z You are in 'detached HEAD' state. You can look around, make experimental 2025-12-04T09:21:06.4939868Z changes and commit them, and you can discard any commits you make in this 2025-12-04T09:21:06.4940514Z state without impacting any branches by switching back to a branch. 2025-12-04T09:21:06.4940927Z 2025-12-04T09:21:06.4941174Z If you want to create a new branch to retain commits you create, you may 2025-12-04T09:21:06.4941768Z do so (now or later) by using -c with the switch command. Example: 2025-12-04T09:21:06.4942108Z 2025-12-04T09:21:06.4942252Z git switch -c 2025-12-04T09:21:06.4942485Z 2025-12-04T09:21:06.4942626Z Or undo this operation with: 2025-12-04T09:21:06.4942837Z 2025-12-04T09:21:06.4942945Z git switch - 2025-12-04T09:21:06.4943113Z 2025-12-04T09:21:06.4943389Z Turn off this advice by setting config variable advice.detachedHead to false 2025-12-04T09:21:06.4943809Z 2025-12-04T09:21:06.4948283Z HEAD is now at ffd9b0fb435 Resolve collective autotuning test failure on arm (#168919) 2025-12-04T09:21:06.5027640Z ##[endgroup] 2025-12-04T09:21:06.5028361Z ##[group]Setting up auth for fetching submodules 2025-12-04T09:21:06.5034556Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2025-12-04T09:21:06.5091673Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2025-12-04T09:21:06.5120804Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2025-12-04T09:21:06.5154460Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2025-12-04T09:21:06.5180073Z ##[endgroup] 2025-12-04T09:21:06.5180522Z ##[group]Fetching submodules 2025-12-04T09:21:06.5183446Z [command]/usr/bin/git submodule sync --recursive 2025-12-04T09:21:06.5553454Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2025-12-04T09:21:06.5914548Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2025-12-04T09:21:06.5915969Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2025-12-04T09:21:06.5919123Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2025-12-04T09:21:06.5922216Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2025-12-04T09:21:06.5924905Z Submodule 'third_party/NVTX' (https://github.com/NVIDIA/NVTX.git) registered for path 'third_party/NVTX' 2025-12-04T09:21:06.5928712Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2025-12-04T09:21:06.5931480Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2025-12-04T09:21:06.5934763Z Submodule 'third_party/aiter' (https://github.com/ROCm/aiter.git) registered for path 'third_party/aiter' 2025-12-04T09:21:06.5938268Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2025-12-04T09:21:06.5942125Z Submodule 'third_party/composable_kernel' (https://github.com/ROCm/composable_kernel.git) registered for path 'third_party/composable_kernel' 2025-12-04T09:21:06.5945634Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2025-12-04T09:21:06.5949882Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2025-12-04T09:21:06.5953982Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2025-12-04T09:21:06.5957915Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2025-12-04T09:21:06.5968160Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2025-12-04T09:21:06.5974359Z Submodule 'third_party/flash-attention' (https://github.com/Dao-AILab/flash-attention.git) registered for path 'third_party/flash-attention' 2025-12-04T09:21:06.5980418Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2025-12-04T09:21:06.5984831Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2025-12-04T09:21:06.5989999Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:21:06.5994593Z Submodule 'third_party/gloo' (https://github.com/pytorch/gloo) registered for path 'third_party/gloo' 2025-12-04T09:21:06.5999677Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2025-12-04T09:21:06.6005362Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2025-12-04T09:21:06.6010439Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2025-12-04T09:21:06.6015640Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2025-12-04T09:21:06.6021045Z Submodule 'third_party/kleidiai' (https://github.com/ARM-software/kleidiai.git) registered for path 'third_party/kleidiai' 2025-12-04T09:21:06.6032437Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2025-12-04T09:21:06.6033644Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2025-12-04T09:21:06.6037980Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2025-12-04T09:21:06.6043915Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2025-12-04T09:21:06.6049328Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2025-12-04T09:21:06.6055109Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2025-12-04T09:21:06.6060941Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2025-12-04T09:21:06.6067272Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2025-12-04T09:21:06.6075181Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2025-12-04T09:21:06.6081711Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2025-12-04T09:21:06.6088545Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2025-12-04T09:21:06.6095240Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2025-12-04T09:21:06.6131374Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2025-12-04T09:21:06.8953788Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2025-12-04T09:21:06.8954965Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2025-12-04T09:21:06.8956339Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2025-12-04T09:21:06.8957385Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2025-12-04T09:21:06.8959020Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2025-12-04T09:21:06.8959936Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2025-12-04T09:21:06.9784153Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NVTX'... 2025-12-04T09:21:07.3687565Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2025-12-04T09:21:07.3689736Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2025-12-04T09:21:07.3691687Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2025-12-04T09:21:07.3693601Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2025-12-04T09:21:07.3695459Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2025-12-04T09:21:07.3697615Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2025-12-04T09:21:07.3699659Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2025-12-04T09:21:07.4689357Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2025-12-04T09:21:08.9745963Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kleidiai'... 2025-12-04T09:21:08.9747026Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2025-12-04T09:21:08.9747920Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2025-12-04T09:21:08.9748844Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flash-attention'... 2025-12-04T09:21:08.9749738Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2025-12-04T09:21:08.9750580Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2025-12-04T09:21:08.9751465Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2025-12-04T09:21:08.9752323Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2025-12-04T09:21:08.9753175Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2025-12-04T09:21:08.9754063Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2025-12-04T09:21:08.9754940Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2025-12-04T09:21:08.9755809Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2025-12-04T09:21:09.0748137Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2025-12-04T09:21:22.8674896Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2025-12-04T09:21:22.8676127Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2025-12-04T09:21:22.8676952Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2025-12-04T09:21:22.8677851Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/composable_kernel'... 2025-12-04T09:21:22.8678996Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2025-12-04T09:21:22.9675613Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2025-12-04T09:21:22.9676551Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2025-12-04T09:21:22.9677403Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/aiter'... 2025-12-04T09:21:25.2637435Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2025-12-04T09:21:25.2782941Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2025-12-04T09:21:25.2906488Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2025-12-04T09:21:25.3204785Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2025-12-04T09:21:25.4204192Z Submodule path 'third_party/NVTX': checked out '3ebbc93ded7285963bff932c678fa367eb393ba6' 2025-12-04T09:21:25.4875724Z Submodule path 'third_party/VulkanMemoryAllocator': checked out '1d8f600fd424278486eade7ed3e877c99f0846b1' 2025-12-04T09:21:26.3653272Z Submodule path 'third_party/XNNPACK': checked out '51a0103656eff6fc9bfd39a4597923c4b542c883' 2025-12-04T09:21:26.5886315Z Submodule path 'third_party/aiter': checked out '01aae101b9e5e94d6c16a9514c9fb8df99c93150' 2025-12-04T09:21:26.5909303Z Submodule '3rdparty/composable_kernel' (https://github.com/ROCm/composable_kernel.git) registered for path 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:21:26.5942498Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/aiter/3rdparty/composable_kernel'... 2025-12-04T09:21:31.7088562Z Submodule path 'third_party/aiter/3rdparty/composable_kernel': checked out 'cffe8fa2a442ac8e80dd236a1a5d24fe3d7e0cbf' 2025-12-04T09:21:31.7388727Z Submodule path 'third_party/benchmark': checked out '299e5928955cc62af9968370293b916f5130916f' 2025-12-04T09:21:32.1630407Z Submodule path 'third_party/composable_kernel': checked out '7fe50dc3da2069d6645d9deb8c017a876472a977' 2025-12-04T09:21:32.2230565Z Submodule path 'third_party/cpp-httplib': checked out '89c932f313c6437c38f2982869beacc89c2f2246' 2025-12-04T09:21:32.3384325Z Submodule path 'third_party/cpuinfo': checked out 'f858c30bcb16f8effd5ff46996f0514539e17abc' 2025-12-04T09:21:32.3968254Z Submodule path 'third_party/cudnn_frontend': checked out '0b1577c8c83401237d601d0d0db5210506705396' 2025-12-04T09:21:33.1691211Z Submodule path 'third_party/cutlass': checked out 'f88806b1e31dfa579842638740216dd41fc6c588' 2025-12-04T09:21:33.3553803Z Submodule path 'third_party/fbgemm': checked out 'c0b988d39a9e47c794d699f29930ed4d7c7e13a4' 2025-12-04T09:21:33.3579803Z Submodule 'external/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/external/asmjit' 2025-12-04T09:21:33.3581225Z Submodule 'external/composable_kernel' (https://github.com/ROCm/composable_kernel.git) registered for path 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:21:33.3583755Z Submodule 'external/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:21:33.3586977Z Submodule 'external/cutlass' (https://github.com/jwfromm/cutlass) registered for path 'third_party/fbgemm/external/cutlass' 2025-12-04T09:21:33.3590413Z Submodule 'external/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/external/googletest' 2025-12-04T09:21:33.3593902Z Submodule 'external/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:21:33.3597139Z Submodule 'external/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/fbgemm/external/json' 2025-12-04T09:21:33.3629631Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/asmjit'... 2025-12-04T09:21:34.4755373Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/hipify_torch'... 2025-12-04T09:21:34.4757449Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/cpuinfo'... 2025-12-04T09:21:34.4759579Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/googletest'... 2025-12-04T09:21:34.5756491Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/composable_kernel'... 2025-12-04T09:21:38.0278911Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/cutlass'... 2025-12-04T09:21:38.1280446Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/external/json'... 2025-12-04T09:21:40.8841975Z Submodule path 'third_party/fbgemm/external/asmjit': checked out 'a3199e8857792cd10b7589ff5d58343d2c9008ea' 2025-12-04T09:21:41.3098250Z Submodule path 'third_party/fbgemm/external/composable_kernel': checked out '7fe50dc3da2069d6645d9deb8c017a876472a977' 2025-12-04T09:21:41.4306896Z Submodule path 'third_party/fbgemm/external/cpuinfo': checked out '6543fec09b2f04ac4a666882998b534afc9c1349' 2025-12-04T09:21:42.2069251Z Submodule path 'third_party/fbgemm/external/cutlass': checked out '98125ce499b0fdf7ffbe0e3052f5b8709f4840f8' 2025-12-04T09:21:42.2624791Z Submodule path 'third_party/fbgemm/external/googletest': checked out '52eb8108c5bdec04579160ae17225d66034bd723' 2025-12-04T09:21:42.2770213Z Submodule path 'third_party/fbgemm/external/hipify_torch': checked out '63b6a7b541fa7f08f8475ca7d74054db36ff2691' 2025-12-04T09:21:42.3987935Z Submodule path 'third_party/fbgemm/external/json': checked out '9cca280a4d0ccf0c08f47a99aa71d1b0e52f8d03' 2025-12-04T09:21:42.4840823Z Submodule path 'third_party/flash-attention': checked out '979702c87a8713a8e0a5e9fee122b90d2ef13be5' 2025-12-04T09:21:42.4860387Z Submodule 'csrc/composable_kernel' (https://github.com/ROCm/composable_kernel.git) registered for path 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:21:42.4862447Z Submodule 'csrc/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:21:42.4894681Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flash-attention/csrc/composable_kernel'... 2025-12-04T09:21:47.0721036Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flash-attention/csrc/cutlass'... 2025-12-04T09:21:47.3651993Z Submodule path 'third_party/flash-attention/csrc/composable_kernel': checked out '888317e698e9803c62bd38568abc9e05d7709f33' 2025-12-04T09:21:48.0408601Z Submodule path 'third_party/flash-attention/csrc/cutlass': checked out 'c506e16788cb08416a4a57e11a9067beeee29420' 2025-12-04T09:21:48.2077479Z Submodule path 'third_party/flatbuffers': checked out 'a2cd1ea3b6d3fee220106b5fed3f7ce8da9eb757' 2025-12-04T09:21:48.2436192Z Submodule path 'third_party/fmt': checked out '407c905e45ad75fc29bf0f9bb7c5c2fd3475976f' 2025-12-04T09:21:48.2928330Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2025-12-04T09:21:48.3249246Z Submodule path 'third_party/gloo': checked out '54cbae0d3a67fa890b4c3d9ee162b7860315e341' 2025-12-04T09:21:48.3798224Z Submodule path 'third_party/googletest': checked out '52eb8108c5bdec04579160ae17225d66034bd723' 2025-12-04T09:21:48.3952729Z Submodule path 'third_party/ideep': checked out '719d8e6cd7f7a0e01b155657526d693acf97c2b3' 2025-12-04T09:21:48.3978099Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2025-12-04T09:21:48.4005915Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2025-12-04T09:22:04.3832574Z Submodule path 'third_party/ideep/mkl-dnn': checked out '8d263e693366ef8db40acc569cc7d8edf644556d' 2025-12-04T09:22:04.4083486Z Submodule path 'third_party/ittapi': checked out 'dec1d23ca65ab069d225dfe40dea14f455170959' 2025-12-04T09:22:04.5098553Z Submodule path 'third_party/kineto': checked out '31f85df8fbd89c188f14ef10f1ec65379786b943' 2025-12-04T09:22:04.5125586Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:22:04.5127569Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:22:04.5131004Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:22:04.5163739Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2025-12-04T09:22:05.8229585Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2025-12-04T09:22:06.0374165Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2025-12-04T09:22:06.1462062Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out 'd2ffe0a4e3acace628db49974246b66fc3e85fb1' 2025-12-04T09:22:06.1485489Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:22:06.1489122Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:22:06.1490762Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:22:06.1494257Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:22:06.1497673Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:22:06.1502430Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:22:06.1507085Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:22:06.1511207Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:22:06.1515152Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:22:06.1548964Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2025-12-04T09:22:07.9931613Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2025-12-04T09:22:07.9933363Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp'... 2025-12-04T09:22:07.9934929Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2025-12-04T09:22:07.9936442Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2025-12-04T09:22:07.9938198Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2025-12-04T09:22:07.9939691Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2025-12-04T09:22:07.9941280Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2025-12-04T09:22:08.0929978Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2025-12-04T09:22:14.8503764Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2025-12-04T09:22:14.8733798Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2025-12-04T09:22:14.9154937Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2025-12-04T09:22:14.9331193Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2025-12-04T09:22:14.9349914Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:22:14.9388996Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2025-12-04T09:22:15.2427751Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2025-12-04T09:22:15.2668159Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2025-12-04T09:22:15.3206086Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '52eb8108c5bdec04579160ae17225d66034bd723' 2025-12-04T09:22:15.4421512Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2025-12-04T09:22:15.4622097Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2025-12-04T09:22:15.4830169Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp': checked out 'b1234816facfdda29845c46696a02998a4af115a' 2025-12-04T09:22:15.4855662Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:15.4859982Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:15.4889888Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb'... 2025-12-04T09:22:17.5197878Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest'... 2025-12-04T09:22:17.8191493Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'd7ba35bbb649209c66e582d5a0244ba988a15159' 2025-12-04T09:22:17.8772884Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2025-12-04T09:22:17.9165614Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '40626af88bd7df9a5fb80be7b25ac85b122d6c21' 2025-12-04T09:22:17.9715410Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '52eb8108c5bdec04579160ae17225d66034bd723' 2025-12-04T09:22:18.0388401Z Submodule path 'third_party/kleidiai': checked out 'd7770c89632329a9914ef1a90289917597639cbe' 2025-12-04T09:22:18.0863975Z Submodule path 'third_party/mimalloc': checked out 'fbd8b99c2b828428947d70fdc046bb55609be93e' 2025-12-04T09:22:18.2077356Z Submodule path 'third_party/nlohmann': checked out '55f93686c01528224f448c19128836e7df245f72' 2025-12-04T09:22:18.7063719Z Submodule path 'third_party/onnx': checked out 'e709452ef2bbc1d113faf678c24e6d3467696e83' 2025-12-04T09:22:18.7103691Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2025-12-04T09:22:18.7135768Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2025-12-04T09:22:19.5691688Z Submodule path 'third_party/onnx/third_party/pybind11': checked out 'a2e59f0e7065404b44dfe92a28aca47ba1378dc4' 2025-12-04T09:22:19.6537842Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2025-12-04T09:22:19.6557277Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:22:19.6559621Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:22:19.6562681Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:22:19.6565893Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:22:19.6569073Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:22:19.6572213Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:22:19.6575648Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:22:19.6578813Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:22:19.6618379Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2025-12-04T09:22:20.0595228Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2025-12-04T09:22:20.0596801Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2025-12-04T09:22:20.0598312Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2025-12-04T09:22:20.0599696Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2025-12-04T09:22:20.1596769Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2025-12-04T09:22:20.7447354Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2025-12-04T09:22:27.4026985Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2025-12-04T09:22:28.1421272Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2025-12-04T09:22:28.1917443Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2025-12-04T09:22:28.2124362Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2025-12-04T09:22:28.3366507Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2025-12-04T09:22:28.3548025Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2025-12-04T09:22:28.3738234Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2025-12-04T09:22:28.3931381Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2025-12-04T09:22:28.3950233Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:28.3952603Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:28.3986169Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2025-12-04T09:22:30.3402031Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2025-12-04T09:22:30.6378300Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2025-12-04T09:22:30.6942901Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2025-12-04T09:22:31.2688163Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2025-12-04T09:22:31.2831313Z Submodule path 'third_party/pocketfft': checked out '0fa0ef591e38c2758e3184c6c23e497b9f732ffa' 2025-12-04T09:22:31.6046471Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2025-12-04T09:22:31.6071486Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:22:31.6078793Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2025-12-04T09:22:31.6111421Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2025-12-04T09:22:32.1672214Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2025-12-04T09:22:32.5105434Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2025-12-04T09:22:32.5994941Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2025-12-04T09:22:32.6127607Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2025-12-04T09:22:32.6274469Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2025-12-04T09:22:32.6788226Z Submodule path 'third_party/pybind11': checked out 'f5fbe867d2d26e4a0a9177a51f6e568868ad3dc8' 2025-12-04T09:22:32.7142394Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2025-12-04T09:22:32.7662184Z Submodule path 'third_party/sleef': checked out '5a1d179df9cf652951b59010a2d2075372d67f68' 2025-12-04T09:22:32.7998032Z Submodule path 'third_party/tensorpipe': checked out '2b4cd91092d335a697416b2a3cb398283246849d' 2025-12-04T09:22:32.8027009Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:22:32.8028595Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:22:32.8030115Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:22:32.8032870Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:22:32.8063628Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2025-12-04T09:22:33.8066945Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2025-12-04T09:22:33.8068365Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2025-12-04T09:22:33.8603848Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2025-12-04T09:22:33.9294491Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2025-12-04T09:22:33.9491477Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2025-12-04T09:22:34.0383585Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '5152db2cbfeb5582e9c27c5ea1dba2cd9e10759b' 2025-12-04T09:22:34.0733773Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2025-12-04T09:22:34.0751018Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:22:34.0788801Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2025-12-04T09:22:34.2771217Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2025-12-04T09:22:34.2811654Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2025-12-04T09:22:34.3173663Z Entering 'android/libs/fbjni' 2025-12-04T09:22:34.3223748Z Entering 'third_party/FP16' 2025-12-04T09:22:34.3272281Z Entering 'third_party/FXdiv' 2025-12-04T09:22:34.3326001Z Entering 'third_party/NNPACK' 2025-12-04T09:22:34.3371913Z Entering 'third_party/NVTX' 2025-12-04T09:22:34.3426265Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:22:34.3474848Z Entering 'third_party/XNNPACK' 2025-12-04T09:22:34.3545772Z Entering 'third_party/aiter' 2025-12-04T09:22:34.3598529Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:22:34.3659071Z Entering 'third_party/benchmark' 2025-12-04T09:22:34.3713904Z Entering 'third_party/composable_kernel' 2025-12-04T09:22:34.3775440Z Entering 'third_party/cpp-httplib' 2025-12-04T09:22:34.3826445Z Entering 'third_party/cpuinfo' 2025-12-04T09:22:34.3874793Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:22:34.3930119Z Entering 'third_party/cutlass' 2025-12-04T09:22:34.3988979Z Entering 'third_party/fbgemm' 2025-12-04T09:22:34.4047297Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:22:34.4094312Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:22:34.4154201Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:22:34.4201930Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:22:34.4265219Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:22:34.4311695Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:22:34.4366637Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:22:34.4418123Z Entering 'third_party/flash-attention' 2025-12-04T09:22:34.4468617Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:22:34.4523639Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:22:34.4582383Z Entering 'third_party/flatbuffers' 2025-12-04T09:22:34.4638172Z Entering 'third_party/fmt' 2025-12-04T09:22:34.4688751Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:22:34.4743975Z Entering 'third_party/gloo' 2025-12-04T09:22:34.4795794Z Entering 'third_party/googletest' 2025-12-04T09:22:34.4851933Z Entering 'third_party/ideep' 2025-12-04T09:22:34.4896156Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:22:34.4952956Z Entering 'third_party/ittapi' 2025-12-04T09:22:34.5003170Z Entering 'third_party/kineto' 2025-12-04T09:22:34.5054463Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:22:34.5100165Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:22:34.5150425Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:22:34.5203809Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:22:34.5247906Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:22:34.5298336Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:22:34.5347043Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:22:34.5404235Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:22:34.5456092Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:22:34.5510624Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:22:34.5564351Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:22:34.5618666Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:34.5669167Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:34.5733263Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:22:34.5780555Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:22:34.5828093Z Entering 'third_party/kleidiai' 2025-12-04T09:22:34.5878115Z Entering 'third_party/mimalloc' 2025-12-04T09:22:34.5927046Z Entering 'third_party/nlohmann' 2025-12-04T09:22:34.5976526Z Entering 'third_party/onnx' 2025-12-04T09:22:34.6050930Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:22:34.6106116Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:22:34.6156123Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:22:34.6203022Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:22:34.6249350Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:22:34.6301307Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:22:34.6351664Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:22:34.6403802Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:22:34.6450053Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:22:34.6495429Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:34.6548481Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:34.6605517Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:22:34.6674196Z Entering 'third_party/pocketfft' 2025-12-04T09:22:34.6729959Z Entering 'third_party/protobuf' 2025-12-04T09:22:34.6782039Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:22:34.6831280Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:22:34.6879498Z Entering 'third_party/psimd' 2025-12-04T09:22:34.6934590Z Entering 'third_party/pthreadpool' 2025-12-04T09:22:34.6981077Z Entering 'third_party/pybind11' 2025-12-04T09:22:34.7027200Z Entering 'third_party/python-peachpy' 2025-12-04T09:22:34.7076361Z Entering 'third_party/sleef' 2025-12-04T09:22:34.7125774Z Entering 'third_party/tensorpipe' 2025-12-04T09:22:34.7177386Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:22:34.7228356Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:22:34.7278647Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:22:34.7327186Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:22:34.7372552Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:22:34.7437360Z ##[endgroup] 2025-12-04T09:22:34.7437888Z ##[group]Persisting credentials for submodules 2025-12-04T09:22:34.7448266Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2025-12-04T09:22:34.7811388Z Entering 'android/libs/fbjni' 2025-12-04T09:22:34.7876253Z Entering 'third_party/FP16' 2025-12-04T09:22:34.7942876Z Entering 'third_party/FXdiv' 2025-12-04T09:22:34.8013372Z Entering 'third_party/NNPACK' 2025-12-04T09:22:34.8078612Z Entering 'third_party/NVTX' 2025-12-04T09:22:34.8142797Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:22:34.8208035Z Entering 'third_party/XNNPACK' 2025-12-04T09:22:34.8294331Z Entering 'third_party/aiter' 2025-12-04T09:22:34.8360832Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:22:34.8438560Z Entering 'third_party/benchmark' 2025-12-04T09:22:34.8512012Z Entering 'third_party/composable_kernel' 2025-12-04T09:22:34.8594884Z Entering 'third_party/cpp-httplib' 2025-12-04T09:22:34.8660566Z Entering 'third_party/cpuinfo' 2025-12-04T09:22:34.8724282Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:22:34.8788874Z Entering 'third_party/cutlass' 2025-12-04T09:22:34.8873854Z Entering 'third_party/fbgemm' 2025-12-04T09:22:34.8948294Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:22:34.9011619Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:22:34.9085283Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:22:34.9148412Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:22:34.9222333Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:22:34.9290417Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:22:34.9354137Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:22:34.9425307Z Entering 'third_party/flash-attention' 2025-12-04T09:22:34.9494960Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:22:34.9565770Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:22:34.9641335Z Entering 'third_party/flatbuffers' 2025-12-04T09:22:34.9711152Z Entering 'third_party/fmt' 2025-12-04T09:22:34.9779202Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:22:34.9847251Z Entering 'third_party/gloo' 2025-12-04T09:22:34.9914279Z Entering 'third_party/googletest' 2025-12-04T09:22:34.9984546Z Entering 'third_party/ideep' 2025-12-04T09:22:35.0051087Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:22:35.0128279Z Entering 'third_party/ittapi' 2025-12-04T09:22:35.0197778Z Entering 'third_party/kineto' 2025-12-04T09:22:35.0269051Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:22:35.0330421Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:22:35.0393136Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:22:35.0465254Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:22:35.0529558Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:22:35.0590483Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:22:35.0661691Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:22:35.0726438Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:22:35.0787212Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:22:35.0851598Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:22:35.0916654Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:22:35.0982799Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:35.1050453Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:35.1117507Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:22:35.1181955Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:22:35.1248331Z Entering 'third_party/kleidiai' 2025-12-04T09:22:35.1314854Z Entering 'third_party/mimalloc' 2025-12-04T09:22:35.1387430Z Entering 'third_party/nlohmann' 2025-12-04T09:22:35.1451016Z Entering 'third_party/onnx' 2025-12-04T09:22:35.1535755Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:22:35.1605784Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:22:35.1674694Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:22:35.1741403Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:22:35.1802706Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:22:35.1869857Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:22:35.1938854Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:22:35.2011279Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:22:35.2085071Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:22:35.2154468Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:35.2221273Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:35.2284969Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:22:35.2374422Z Entering 'third_party/pocketfft' 2025-12-04T09:22:35.2448901Z Entering 'third_party/protobuf' 2025-12-04T09:22:35.2508542Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:22:35.2574001Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:22:35.2639774Z Entering 'third_party/psimd' 2025-12-04T09:22:35.2704334Z Entering 'third_party/pthreadpool' 2025-12-04T09:22:35.2768105Z Entering 'third_party/pybind11' 2025-12-04T09:22:35.2835582Z Entering 'third_party/python-peachpy' 2025-12-04T09:22:35.2900411Z Entering 'third_party/sleef' 2025-12-04T09:22:35.2975859Z Entering 'third_party/tensorpipe' 2025-12-04T09:22:35.3044851Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:22:35.3107515Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:22:35.3176263Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:22:35.3243697Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:22:35.3304174Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:22:35.3393750Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2025-12-04T09:22:35.3751713Z Entering 'android/libs/fbjni' 2025-12-04T09:22:35.3811987Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2025-12-04T09:22:35.3833522Z Entering 'third_party/FP16' 2025-12-04T09:22:35.3895445Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2025-12-04T09:22:35.3914859Z Entering 'third_party/FXdiv' 2025-12-04T09:22:35.3974069Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2025-12-04T09:22:35.3990639Z Entering 'third_party/NNPACK' 2025-12-04T09:22:35.4051290Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2025-12-04T09:22:35.4074938Z Entering 'third_party/NVTX' 2025-12-04T09:22:35.4137202Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config remote.origin.url 2025-12-04T09:22:35.4153360Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:22:35.4212264Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2025-12-04T09:22:35.4231294Z Entering 'third_party/XNNPACK' 2025-12-04T09:22:35.4289776Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2025-12-04T09:22:35.4325302Z Entering 'third_party/aiter' 2025-12-04T09:22:35.4387259Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/config remote.origin.url 2025-12-04T09:22:35.4403254Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:22:35.4462570Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/modules/3rdparty/composable_kernel/config remote.origin.url 2025-12-04T09:22:35.4490506Z Entering 'third_party/benchmark' 2025-12-04T09:22:35.4554703Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:22:35.4579409Z Entering 'third_party/composable_kernel' 2025-12-04T09:22:35.4639159Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config remote.origin.url 2025-12-04T09:22:35.4668253Z Entering 'third_party/cpp-httplib' 2025-12-04T09:22:35.4730433Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2025-12-04T09:22:35.4748689Z Entering 'third_party/cpuinfo' 2025-12-04T09:22:35.4810433Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2025-12-04T09:22:35.4830015Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:22:35.4891482Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2025-12-04T09:22:35.4910829Z Entering 'third_party/cutlass' 2025-12-04T09:22:35.4970148Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2025-12-04T09:22:35.5002476Z Entering 'third_party/fbgemm' 2025-12-04T09:22:35.5060606Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2025-12-04T09:22:35.5082279Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:22:35.5146430Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/asmjit/config remote.origin.url 2025-12-04T09:22:35.5165010Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:22:35.5228260Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/composable_kernel/config remote.origin.url 2025-12-04T09:22:35.5258666Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:22:35.5321496Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cpuinfo/config remote.origin.url 2025-12-04T09:22:35.5340518Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:22:35.5398923Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cutlass/config remote.origin.url 2025-12-04T09:22:35.5429870Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:22:35.5490432Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/googletest/config remote.origin.url 2025-12-04T09:22:35.5517633Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:22:35.5575806Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/hipify_torch/config remote.origin.url 2025-12-04T09:22:35.5592389Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:22:35.5656934Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/json/config remote.origin.url 2025-12-04T09:22:35.5680962Z Entering 'third_party/flash-attention' 2025-12-04T09:22:35.5742726Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config remote.origin.url 2025-12-04T09:22:35.5765268Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:22:35.5823692Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config remote.origin.url 2025-12-04T09:22:35.5851649Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:22:35.5914783Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config remote.origin.url 2025-12-04T09:22:35.5949167Z Entering 'third_party/flatbuffers' 2025-12-04T09:22:35.6008450Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2025-12-04T09:22:35.6031944Z Entering 'third_party/fmt' 2025-12-04T09:22:35.6095661Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2025-12-04T09:22:35.6113747Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:22:35.6173972Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2025-12-04T09:22:35.6191659Z Entering 'third_party/gloo' 2025-12-04T09:22:35.6261655Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2025-12-04T09:22:35.6281768Z Entering 'third_party/googletest' 2025-12-04T09:22:35.6342455Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:22:35.6363604Z Entering 'third_party/ideep' 2025-12-04T09:22:35.6420747Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2025-12-04T09:22:35.6437862Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:22:35.6497225Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2025-12-04T09:22:35.6528171Z Entering 'third_party/ittapi' 2025-12-04T09:22:35.6589935Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2025-12-04T09:22:35.6609080Z Entering 'third_party/kineto' 2025-12-04T09:22:35.6671291Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2025-12-04T09:22:35.6688804Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:22:35.6758945Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2025-12-04T09:22:35.6778136Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:22:35.6842678Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2025-12-04T09:22:35.6863080Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:22:35.6923392Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2025-12-04T09:22:35.6939962Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:22:35.7002012Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2025-12-04T09:22:35.7021305Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:22:35.7081035Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2025-12-04T09:22:35.7099101Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:22:35.7163789Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2025-12-04T09:22:35.7184507Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:22:35.7245985Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2025-12-04T09:22:35.7265994Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:22:35.7329647Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:22:35.7348960Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:22:35.7412950Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2025-12-04T09:22:35.7432781Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:22:35.7502111Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2025-12-04T09:22:35.7526455Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:22:35.7589859Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/config remote.origin.url 2025-12-04T09:22:35.7607487Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:35.7668195Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2025-12-04T09:22:35.7690937Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:35.7751946Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2025-12-04T09:22:35.7787439Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:22:35.7850842Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2025-12-04T09:22:35.7869022Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:22:35.7927568Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2025-12-04T09:22:35.7951828Z Entering 'third_party/kleidiai' 2025-12-04T09:22:35.8010979Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config remote.origin.url 2025-12-04T09:22:35.8031606Z Entering 'third_party/mimalloc' 2025-12-04T09:22:35.8094412Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2025-12-04T09:22:35.8113260Z Entering 'third_party/nlohmann' 2025-12-04T09:22:35.8173435Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2025-12-04T09:22:35.8192119Z Entering 'third_party/onnx' 2025-12-04T09:22:35.8261276Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2025-12-04T09:22:35.8299447Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:22:35.8366024Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:22:35.8386100Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:22:35.8446471Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2025-12-04T09:22:35.8468309Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:22:35.8537443Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:22:35.8554234Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:22:35.8622037Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:22:35.8638868Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:22:35.8699344Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2025-12-04T09:22:35.8716630Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:22:35.8776679Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2025-12-04T09:22:35.8794792Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:22:35.8855330Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2025-12-04T09:22:35.8872279Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:22:35.8942183Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2025-12-04T09:22:35.8965358Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:22:35.9026109Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2025-12-04T09:22:35.9045113Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:35.9108435Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2025-12-04T09:22:35.9133813Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:35.9196414Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2025-12-04T09:22:35.9218372Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:22:35.9275158Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2025-12-04T09:22:35.9315480Z Entering 'third_party/pocketfft' 2025-12-04T09:22:35.9382938Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2025-12-04T09:22:35.9403075Z Entering 'third_party/protobuf' 2025-12-04T09:22:35.9471191Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2025-12-04T09:22:35.9493740Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:22:35.9551888Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:22:35.9572899Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:22:35.9637227Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:22:35.9659433Z Entering 'third_party/psimd' 2025-12-04T09:22:35.9717803Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2025-12-04T09:22:35.9739053Z Entering 'third_party/pthreadpool' 2025-12-04T09:22:35.9803627Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2025-12-04T09:22:35.9825298Z Entering 'third_party/pybind11' 2025-12-04T09:22:35.9884403Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:22:35.9903332Z Entering 'third_party/python-peachpy' 2025-12-04T09:22:35.9963272Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2025-12-04T09:22:35.9981921Z Entering 'third_party/sleef' 2025-12-04T09:22:36.0042507Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2025-12-04T09:22:36.0062669Z Entering 'third_party/tensorpipe' 2025-12-04T09:22:36.0132162Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2025-12-04T09:22:36.0154830Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:22:36.0215289Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:22:36.0232936Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:22:36.0291558Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2025-12-04T09:22:36.0309292Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:22:36.0373277Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2025-12-04T09:22:36.0389980Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:22:36.0448599Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:22:36.0466386Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:22:36.0531175Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2025-12-04T09:22:36.1174088Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2025-12-04T09:22:36.1535548Z Entering 'android/libs/fbjni' 2025-12-04T09:22:36.1592967Z Entering 'third_party/FP16' 2025-12-04T09:22:36.1644091Z Entering 'third_party/FXdiv' 2025-12-04T09:22:36.1688981Z Entering 'third_party/NNPACK' 2025-12-04T09:22:36.1736710Z Entering 'third_party/NVTX' 2025-12-04T09:22:36.1784636Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:22:36.1831549Z Entering 'third_party/XNNPACK' 2025-12-04T09:22:36.1900317Z Entering 'third_party/aiter' 2025-12-04T09:22:36.1950509Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:22:36.2012100Z Entering 'third_party/benchmark' 2025-12-04T09:22:36.2062282Z Entering 'third_party/composable_kernel' 2025-12-04T09:22:36.2120353Z Entering 'third_party/cpp-httplib' 2025-12-04T09:22:36.2175699Z Entering 'third_party/cpuinfo' 2025-12-04T09:22:36.2223941Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:22:36.2269837Z Entering 'third_party/cutlass' 2025-12-04T09:22:36.2330908Z Entering 'third_party/fbgemm' 2025-12-04T09:22:36.2384740Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:22:36.2429573Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:22:36.2490937Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:22:36.2539377Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:22:36.2594105Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:22:36.2649355Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:22:36.2697789Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:22:36.2748170Z Entering 'third_party/flash-attention' 2025-12-04T09:22:36.2796158Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:22:36.2852633Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:22:36.2913125Z Entering 'third_party/flatbuffers' 2025-12-04T09:22:36.2971727Z Entering 'third_party/fmt' 2025-12-04T09:22:36.3019811Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:22:36.3073046Z Entering 'third_party/gloo' 2025-12-04T09:22:36.3119897Z Entering 'third_party/googletest' 2025-12-04T09:22:36.3168838Z Entering 'third_party/ideep' 2025-12-04T09:22:36.3222553Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:22:36.3278565Z Entering 'third_party/ittapi' 2025-12-04T09:22:36.3328140Z Entering 'third_party/kineto' 2025-12-04T09:22:36.3375126Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:22:36.3428807Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:22:36.3475825Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:22:36.3534052Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:22:36.3580776Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:22:36.3628967Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:22:36.3683229Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:22:36.3736643Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:22:36.3784493Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:22:36.3830309Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:22:36.3876502Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:22:36.3928752Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:36.3983885Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:36.4035058Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:22:36.4081345Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:22:36.4135176Z Entering 'third_party/kleidiai' 2025-12-04T09:22:36.4183675Z Entering 'third_party/mimalloc' 2025-12-04T09:22:36.4236211Z Entering 'third_party/nlohmann' 2025-12-04T09:22:36.4289893Z Entering 'third_party/onnx' 2025-12-04T09:22:36.4357616Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:22:36.4410790Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:22:36.4464329Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:22:36.4510639Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:22:36.4556492Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:22:36.4605978Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:22:36.4656537Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:22:36.4702732Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:22:36.4749499Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:22:36.4794038Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:36.4846216Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:36.4892500Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:22:36.4968610Z Entering 'third_party/pocketfft' 2025-12-04T09:22:36.5017946Z Entering 'third_party/protobuf' 2025-12-04T09:22:36.5071772Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:22:36.5116719Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:22:36.5169269Z Entering 'third_party/psimd' 2025-12-04T09:22:36.5218266Z Entering 'third_party/pthreadpool' 2025-12-04T09:22:36.5264397Z Entering 'third_party/pybind11' 2025-12-04T09:22:36.5313128Z Entering 'third_party/python-peachpy' 2025-12-04T09:22:36.5363893Z Entering 'third_party/sleef' 2025-12-04T09:22:36.5409901Z Entering 'third_party/tensorpipe' 2025-12-04T09:22:36.5461773Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:22:36.5511201Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:22:36.5553448Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:22:36.5602645Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:22:36.5652502Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:22:36.5719127Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2025-12-04T09:22:36.6073889Z Entering 'android/libs/fbjni' 2025-12-04T09:22:36.6124088Z Entering 'third_party/FP16' 2025-12-04T09:22:36.6175810Z Entering 'third_party/FXdiv' 2025-12-04T09:22:36.6228440Z Entering 'third_party/NNPACK' 2025-12-04T09:22:36.6277716Z Entering 'third_party/NVTX' 2025-12-04T09:22:36.6325163Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:22:36.6376081Z Entering 'third_party/XNNPACK' 2025-12-04T09:22:36.6440780Z Entering 'third_party/aiter' 2025-12-04T09:22:36.6495180Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:22:36.6555557Z Entering 'third_party/benchmark' 2025-12-04T09:22:36.6610664Z Entering 'third_party/composable_kernel' 2025-12-04T09:22:36.6672835Z Entering 'third_party/cpp-httplib' 2025-12-04T09:22:36.6727632Z Entering 'third_party/cpuinfo' 2025-12-04T09:22:36.6778995Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:22:36.6837316Z Entering 'third_party/cutlass' 2025-12-04T09:22:36.6898982Z Entering 'third_party/fbgemm' 2025-12-04T09:22:36.6949343Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:22:36.6995240Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:22:36.7054238Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:22:36.7103234Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:22:36.7162499Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:22:36.7212216Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:22:36.7264338Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:22:36.7318541Z Entering 'third_party/flash-attention' 2025-12-04T09:22:36.7368845Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:22:36.7427435Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:22:36.7488557Z Entering 'third_party/flatbuffers' 2025-12-04T09:22:36.7540862Z Entering 'third_party/fmt' 2025-12-04T09:22:36.7588807Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:22:36.7644687Z Entering 'third_party/gloo' 2025-12-04T09:22:36.7697629Z Entering 'third_party/googletest' 2025-12-04T09:22:36.7748613Z Entering 'third_party/ideep' 2025-12-04T09:22:36.7798957Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:22:36.7862526Z Entering 'third_party/ittapi' 2025-12-04T09:22:36.7915027Z Entering 'third_party/kineto' 2025-12-04T09:22:36.7972638Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:22:36.8020294Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:22:36.8074213Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:22:36.8124766Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:22:36.8172225Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:22:36.8223310Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:22:36.8271622Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:22:36.8324289Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:22:36.8371889Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:22:36.8424872Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:22:36.8470251Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:22:36.8515692Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:36.8571766Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:36.8626741Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:22:36.8671673Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:22:36.8721965Z Entering 'third_party/kleidiai' 2025-12-04T09:22:36.8770139Z Entering 'third_party/mimalloc' 2025-12-04T09:22:36.8814573Z Entering 'third_party/nlohmann' 2025-12-04T09:22:36.8868692Z Entering 'third_party/onnx' 2025-12-04T09:22:36.8941065Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:22:36.8992330Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:22:36.9043746Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:22:36.9091218Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:22:36.9137865Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:22:36.9187334Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:22:36.9242030Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:22:36.9288532Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:22:36.9339475Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:22:36.9385745Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:36.9431628Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:36.9483547Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:22:36.9557013Z Entering 'third_party/pocketfft' 2025-12-04T09:22:36.9610071Z Entering 'third_party/protobuf' 2025-12-04T09:22:36.9665278Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:22:36.9714147Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:22:36.9760478Z Entering 'third_party/psimd' 2025-12-04T09:22:36.9808194Z Entering 'third_party/pthreadpool' 2025-12-04T09:22:36.9857286Z Entering 'third_party/pybind11' 2025-12-04T09:22:36.9904456Z Entering 'third_party/python-peachpy' 2025-12-04T09:22:36.9953910Z Entering 'third_party/sleef' 2025-12-04T09:22:37.0008046Z Entering 'third_party/tensorpipe' 2025-12-04T09:22:37.0050260Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:22:37.0101368Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:22:37.0150933Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:22:37.0202309Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:22:37.0248548Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:22:37.0310698Z ##[endgroup] 2025-12-04T09:22:37.0352260Z [command]/usr/bin/git log -1 --format=%H 2025-12-04T09:22:37.0383535Z ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:22:37.0493341Z ##[group]Run cd "${GITHUB_WORKSPACE}" 2025-12-04T09:22:37.0493755Z cd "${GITHUB_WORKSPACE}" 2025-12-04T09:22:37.0494231Z # Clean stale submodule dirs 2025-12-04T09:22:37.0494605Z if [ -z "${NO_SUDO}" ]; then 2025-12-04T09:22:37.0495060Z  sudo git submodule foreach --recursive git clean -ffdx 2025-12-04T09:22:37.0495499Z else 2025-12-04T09:22:37.0495848Z  git submodule foreach --recursive git clean -ffdx 2025-12-04T09:22:37.0496268Z fi 2025-12-04T09:22:37.0511825Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:37.0512279Z env: 2025-12-04T09:22:37.0512538Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:37.0512836Z NO_SUDO: true 2025-12-04T09:22:37.0513117Z ##[endgroup] 2025-12-04T09:22:37.0897570Z Entering 'android/libs/fbjni' 2025-12-04T09:22:37.0937858Z Entering 'third_party/FP16' 2025-12-04T09:22:37.0974368Z Entering 'third_party/FXdiv' 2025-12-04T09:22:37.1014188Z Entering 'third_party/NNPACK' 2025-12-04T09:22:37.1055130Z Entering 'third_party/NVTX' 2025-12-04T09:22:37.1102686Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:22:37.1140749Z Entering 'third_party/XNNPACK' 2025-12-04T09:22:37.1284734Z Entering 'third_party/aiter' 2025-12-04T09:22:37.1336447Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:22:37.1459607Z Entering 'third_party/benchmark' 2025-12-04T09:22:37.1498643Z Entering 'third_party/composable_kernel' 2025-12-04T09:22:37.1643508Z Entering 'third_party/cpp-httplib' 2025-12-04T09:22:37.1682955Z Entering 'third_party/cpuinfo' 2025-12-04T09:22:37.1725350Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:22:37.1770037Z Entering 'third_party/cutlass' 2025-12-04T09:22:37.1891898Z Entering 'third_party/fbgemm' 2025-12-04T09:22:37.1965440Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:22:37.2002333Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:22:37.2135965Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:22:37.2175969Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:22:37.2288320Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:22:37.2330958Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:22:37.2365209Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:22:37.2415200Z Entering 'third_party/flash-attention' 2025-12-04T09:22:37.2459367Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:22:37.2570900Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:22:37.2671203Z Entering 'third_party/flatbuffers' 2025-12-04T09:22:37.2751046Z Entering 'third_party/fmt' 2025-12-04T09:22:37.2792618Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:22:37.2840755Z Entering 'third_party/gloo' 2025-12-04T09:22:37.2883913Z Entering 'third_party/googletest' 2025-12-04T09:22:37.2922852Z Entering 'third_party/ideep' 2025-12-04T09:22:37.2962076Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:22:37.3058501Z Entering 'third_party/ittapi' 2025-12-04T09:22:37.3103032Z Entering 'third_party/kineto' 2025-12-04T09:22:37.3152056Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:22:37.3192997Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:22:37.3248057Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:22:37.3288831Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:22:37.3324754Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:22:37.3359120Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:22:37.3402834Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:22:37.3437768Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:22:37.3476630Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:22:37.3530974Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:22:37.3571709Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:22:37.3607751Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:37.3665780Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:37.3708956Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:22:37.3747071Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:22:37.3792822Z Entering 'third_party/kleidiai' 2025-12-04T09:22:37.3843727Z Entering 'third_party/mimalloc' 2025-12-04T09:22:37.3884715Z Entering 'third_party/nlohmann' 2025-12-04T09:22:37.3933771Z Entering 'third_party/onnx' 2025-12-04T09:22:37.4312516Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:22:37.4358191Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:22:37.4423079Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:22:37.4459893Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:22:37.4497211Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:22:37.4533184Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:22:37.4586110Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:22:37.4622258Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:22:37.4662682Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:22:37.4697812Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:22:37.4750467Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:22:37.4794486Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:22:37.5099760Z Entering 'third_party/pocketfft' 2025-12-04T09:22:37.5133706Z Entering 'third_party/protobuf' 2025-12-04T09:22:37.5224356Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:22:37.5259409Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:22:37.5303096Z Entering 'third_party/psimd' 2025-12-04T09:22:37.5341293Z Entering 'third_party/pthreadpool' 2025-12-04T09:22:37.5382954Z Entering 'third_party/pybind11' 2025-12-04T09:22:37.5422078Z Entering 'third_party/python-peachpy' 2025-12-04T09:22:37.5459509Z Entering 'third_party/sleef' 2025-12-04T09:22:37.5497700Z Entering 'third_party/tensorpipe' 2025-12-04T09:22:37.5538603Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:22:37.5574428Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:22:37.5610154Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:22:37.5648898Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:22:37.5683016Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:22:37.5864440Z Prepare all required actions 2025-12-04T09:22:37.5865045Z Getting action download info 2025-12-04T09:22:37.7493618Z ##[group]Run ./.github/actions/setup-linux 2025-12-04T09:22:37.7493975Z env: 2025-12-04T09:22:37.7494227Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:37.7494529Z ##[endgroup] 2025-12-04T09:22:37.7538243Z ##[group]Run set -euo pipefail 2025-12-04T09:22:37.7538664Z set -euo pipefail 2025-12-04T09:22:37.7538995Z function get_ec2_metadata() { 2025-12-04T09:22:37.7539441Z  # Pulled from instance metadata endpoint for EC2 2025-12-04T09:22:37.7540179Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2025-12-04T09:22:37.7540991Z  category=$1 2025-12-04T09:22:37.7541412Z  # If it is GCP runner (runner name contains gcp), do not run this 2025-12-04T09:22:37.7541931Z  runner_name_str=i-01ba3f41dbfc68ae5 2025-12-04T09:22:37.7542360Z  if [[ -f /.inarc ]]; then 2025-12-04T09:22:37.7542765Z  echo "ARC Runner, no info on ec2 metadata" 2025-12-04T09:22:37.7543223Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2025-12-04T09:22:37.7543773Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2025-12-04T09:22:37.7544267Z  else 2025-12-04T09:22:37.7545275Z  curl -H "X-aws-ec2-metadata-token: $(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 30")" -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2025-12-04T09:22:37.7546360Z  fi 2025-12-04T09:22:37.7546609Z } 2025-12-04T09:22:37.7547210Z echo "ami-id: $(get_ec2_metadata ami-id)" 2025-12-04T09:22:37.7547748Z echo "instance-id: $(get_ec2_metadata instance-id)" 2025-12-04T09:22:37.7548320Z echo "instance-type: $(get_ec2_metadata instance-type)" 2025-12-04T09:22:37.7548809Z echo "system info $(uname -a)" 2025-12-04T09:22:37.7555597Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:37.7556051Z env: 2025-12-04T09:22:37.7556290Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:37.7556594Z ##[endgroup] 2025-12-04T09:22:37.7734395Z ami-id: ami-08982f1c5bf93d976 2025-12-04T09:22:37.7853116Z instance-id: i-01ba3f41dbfc68ae5 2025-12-04T09:22:37.7965940Z instance-type: m4.10xlarge 2025-12-04T09:22:37.7976855Z system info Linux ip-10-0-76-7.ec2.internal 6.1.150-174.273.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Sep 9 12:21:26 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux 2025-12-04T09:22:37.7999190Z ##[group]Run if [ -f /usr/bin/nvidia-smi ]; then nvidia-smi; fi 2025-12-04T09:22:37.7999805Z if [ -f /usr/bin/nvidia-smi ]; then nvidia-smi; fi 2025-12-04T09:22:37.8007386Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:37.8007838Z env: 2025-12-04T09:22:37.8008094Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:37.8008388Z ##[endgroup] 2025-12-04T09:22:37.8054520Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:22:37.8055600Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:22:37.8061856Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:37.8062445Z env: 2025-12-04T09:22:37.8062711Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:37.8067869Z ##[endgroup] 2025-12-04T09:22:37.8128359Z ##[group]Run if systemctl is-active --quiet docker; then 2025-12-04T09:22:37.8128893Z if systemctl is-active --quiet docker; then 2025-12-04T09:22:37.8129368Z  echo "Docker daemon is running..."; 2025-12-04T09:22:37.8129837Z else 2025-12-04T09:22:37.8130264Z  echo "Starting docker daemon..." && sudo systemctl start docker; 2025-12-04T09:22:37.8130781Z fi 2025-12-04T09:22:37.8136577Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:37.8137037Z env: 2025-12-04T09:22:37.8137302Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:37.8137727Z ##[endgroup] 2025-12-04T09:22:37.8222488Z Docker daemon is running... 2025-12-04T09:22:37.8270415Z ##[group]Run nick-fields/retry@v3.0.0 2025-12-04T09:22:37.8270764Z with: 2025-12-04T09:22:37.8271007Z shell: bash 2025-12-04T09:22:37.8271274Z timeout_minutes: 5 2025-12-04T09:22:37.8271566Z max_attempts: 3 2025-12-04T09:22:37.8271838Z retry_wait_seconds: 30 2025-12-04T09:22:37.8274646Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2025-12-04T09:22:37.8277615Z polling_interval_seconds: 1 2025-12-04T09:22:37.8277954Z warning_on_retry: true 2025-12-04T09:22:37.8278266Z continue_on_error: false 2025-12-04T09:22:37.8278667Z env: 2025-12-04T09:22:37.8278910Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:37.8279221Z AWS_RETRY_MODE: standard 2025-12-04T09:22:37.8279511Z AWS_MAX_ATTEMPTS: 5 2025-12-04T09:22:37.8279803Z AWS_DEFAULT_REGION: us-east-1 2025-12-04T09:22:37.8280114Z ##[endgroup] 2025-12-04T09:22:39.1800218Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-12-04T09:22:39.1801561Z Configure a credential helper to remove this warning. See 2025-12-04T09:22:39.1802244Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-12-04T09:22:39.1802729Z 2025-12-04T09:22:39.1802849Z Login Succeeded 2025-12-04T09:22:39.9922917Z Command completed after 1 attempt(s). 2025-12-04T09:22:39.9979865Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:22:39.9980463Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:22:39.9981010Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:22:39.9989504Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:39.9989948Z env: 2025-12-04T09:22:39.9990209Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:39.9990525Z ##[endgroup] 2025-12-04T09:22:40.0081334Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2025-12-04T09:22:40.0081990Z # ignore expansion of "docker ps -q" since it could be empty 2025-12-04T09:22:40.0082515Z # shellcheck disable=SC2046 2025-12-04T09:22:40.0082911Z docker stop $(docker ps -q) || true 2025-12-04T09:22:40.0083302Z # Prune all of the docker images 2025-12-04T09:22:40.0083703Z docker system prune -af 2025-12-04T09:22:40.0089898Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:40.0090350Z env: 2025-12-04T09:22:40.0090589Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:40.0090895Z ##[endgroup] 2025-12-04T09:22:40.0573772Z "docker stop" requires at least 1 argument. 2025-12-04T09:22:40.0574258Z See 'docker stop --help'. 2025-12-04T09:22:40.0574469Z 2025-12-04T09:22:40.0574679Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2025-12-04T09:22:40.0574998Z 2025-12-04T09:22:40.0575129Z Stop one or more running containers 2025-12-04T09:22:40.1003937Z Total reclaimed space: 0B 2025-12-04T09:22:40.1225351Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2025-12-04T09:22:40.1225911Z with: 2025-12-04T09:22:40.1227025Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:40.1228338Z use-custom-docker-registry: true 2025-12-04T09:22:40.1228700Z docker-build-dir: .ci/docker 2025-12-04T09:22:40.1229051Z docker-build-script: ./build.sh 2025-12-04T09:22:40.1229403Z working-directory: . 2025-12-04T09:22:40.1229811Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:40.1230273Z force-push: false 2025-12-04T09:22:40.1230541Z env: 2025-12-04T09:22:40.1230786Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:40.1231077Z ##[endgroup] 2025-12-04T09:22:40.1255311Z ##[group]Run set -ex 2025-12-04T09:22:40.1255657Z set -ex 2025-12-04T09:22:40.1255922Z  2025-12-04T09:22:40.1256436Z # If the docker build directory or the build script doesn't exist, the action will 2025-12-04T09:22:40.1257378Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2025-12-04T09:22:40.1258061Z # job could then download the pre-built image as usual 2025-12-04T09:22:40.1258893Z if [[ -d "${DOCKER_BUILD_DIR}" ]] && [[ -f "${DOCKER_BUILD_DIR}/${DOCKER_BUILD_SCRIPT}" ]] && [[ "${USE_CUSTOM_DOCKER_REGISTRY}" == "true" ]]; then 2025-12-04T09:22:40.1259772Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2025-12-04T09:22:40.1260153Z else 2025-12-04T09:22:40.1260449Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2025-12-04T09:22:40.1260971Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:22:40.1261434Z  2025-12-04T09:22:40.1262090Z  echo "Not using custom ECR registry. Either it was not requested or there is no Docker build script in the ${REPO_NAME} repo..." 2025-12-04T09:22:40.1262836Z  exit 0 2025-12-04T09:22:40.1263084Z fi 2025-12-04T09:22:40.1263321Z  2025-12-04T09:22:40.1263707Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2025-12-04T09:22:40.1264409Z  # The docker image name already includes the ECR prefix and tag, so we can just 2025-12-04T09:22:40.1265026Z  # use it as it is, but first let's extract the tag 2025-12-04T09:22:40.1265581Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2025-12-04T09:22:40.1266171Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:22:40.1266834Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:22:40.1267489Z else 2025-12-04T09:22:40.1267796Z  if [[ "${DOCKER_IMAGE_NAME}" == *:* ]]; then 2025-12-04T09:22:40.1268258Z  CUSTOM_TAG_PREFIX=${DOCKER_IMAGE_NAME#*:} 2025-12-04T09:22:40.1268718Z  DOCKER_IMAGE_NAME=${DOCKER_IMAGE_NAME%%:*} 2025-12-04T09:22:40.1269130Z  fi 2025-12-04T09:22:40.1269676Z  DOCKER_TAG=${CUSTOM_TAG_PREFIX:+${CUSTOM_TAG_PREFIX}-}$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2025-12-04T09:22:40.1270396Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:22:40.1271177Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:22:40.1272024Z  echo "custom-tag-prefix=${CUSTOM_TAG_PREFIX}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:22:40.1272537Z fi 2025-12-04T09:22:40.1279113Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:40.1279651Z env: 2025-12-04T09:22:40.1279891Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:40.1280176Z REPO_NAME: pytorch 2025-12-04T09:22:40.1281270Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:40.1282337Z DOCKER_BUILD_DIR: .ci/docker 2025-12-04T09:22:40.1282668Z DOCKER_BUILD_SCRIPT: ./build.sh 2025-12-04T09:22:40.1283083Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:40.1283540Z USE_CUSTOM_DOCKER_REGISTRY: true 2025-12-04T09:22:40.1283869Z CUSTOM_TAG_PREFIX: 2025-12-04T09:22:40.1284124Z ##[endgroup] 2025-12-04T09:22:40.1318097Z + [[ -d .ci/docker ]] 2025-12-04T09:22:40.1318447Z + [[ -f .ci/docker/./build.sh ]] 2025-12-04T09:22:40.1318778Z + [[ true == \t\r\u\e ]] 2025-12-04T09:22:40.1319081Z + echo skip=false 2025-12-04T09:22:40.1320429Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2025-12-04T09:22:40.1323699Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:40.1324908Z ++ awk -F '[:,]' '{print $2}' 2025-12-04T09:22:40.1421298Z + DOCKER_TAG=pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:40.1422581Z + echo docker-tag=pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:40.1424069Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:40.1456955Z ##[group]Run set +e 2025-12-04T09:22:40.1457294Z set +e 2025-12-04T09:22:40.1457543Z set -x 2025-12-04T09:22:40.1457800Z  2025-12-04T09:22:40.1458040Z login() { 2025-12-04T09:22:40.1458578Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2025-12-04T09:22:40.1459197Z } 2025-12-04T09:22:40.1459439Z  2025-12-04T09:22:40.1459674Z retry () { 2025-12-04T09:22:40.1459984Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2025-12-04T09:22:40.1460349Z } 2025-12-04T09:22:40.1460572Z  2025-12-04T09:22:40.1460834Z retry login "${DOCKER_REGISTRY}" 2025-12-04T09:22:40.1461186Z  2025-12-04T09:22:40.1461434Z START_TIME=$(date +%s) 2025-12-04T09:22:40.1461760Z # Wait up to 120 minutes 2025-12-04T09:22:40.1462181Z while [[ $(( $(date +%s) - 7200 )) -lt $START_TIME ]]; do 2025-12-04T09:22:40.1462758Z  # Check if image already exists, if it does then skip building it 2025-12-04T09:22:40.1463322Z  if docker manifest inspect "${DOCKER_IMAGE}"; then 2025-12-04T09:22:40.1463746Z  exit 0 2025-12-04T09:22:40.1464015Z  fi 2025-12-04T09:22:40.1464245Z  2025-12-04T09:22:40.1464697Z  # NB: This flag is used by Docker build workflow to push the image to ECR, so we can 2025-12-04T09:22:40.1465482Z  # use this to differentiate between the Docker build and regular build jobs. For the 2025-12-04T09:22:40.1466261Z  # latter, it will wait for the Docker images to become available before continuing 2025-12-04T09:22:40.1466996Z  if [ "${DOCKER_PUSH:-false}" == "true" ]; then 2025-12-04T09:22:40.1467641Z  # It's a Docker build job, let's build the image 2025-12-04T09:22:40.1468108Z  break 2025-12-04T09:22:40.1468385Z  else 2025-12-04T09:22:40.1468786Z  # It's a regular build job, wait for the image to become available 2025-12-04T09:22:40.1469286Z  sleep 300 2025-12-04T09:22:40.1469576Z  fi 2025-12-04T09:22:40.1469817Z done 2025-12-04T09:22:40.1470065Z  2025-12-04T09:22:40.1470482Z # NB: This part requires a full checkout. Otherwise, the merge base will 2025-12-04T09:22:40.1471309Z # be empty. The default action would be to continue rebuild the image 2025-12-04T09:22:40.1471939Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2025-12-04T09:22:40.1472488Z  # if we're on the base branch then use the parent commit 2025-12-04T09:22:40.1472975Z  MERGE_BASE=$(git rev-parse HEAD~) 2025-12-04T09:22:40.1473340Z else 2025-12-04T09:22:40.1473743Z  # otherwise we're on a PR, so use the most recent base commit 2025-12-04T09:22:40.1474319Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2025-12-04T09:22:40.1474742Z fi 2025-12-04T09:22:40.1474991Z  2025-12-04T09:22:40.1475265Z if [[ -z "${MERGE_BASE}" ]]; then 2025-12-04T09:22:40.1475678Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2025-12-04T09:22:40.1476078Z  2025-12-04T09:22:40.1476640Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2025-12-04T09:22:40.1477293Z  exit 0 2025-12-04T09:22:40.1477655Z fi 2025-12-04T09:22:40.1477905Z  2025-12-04T09:22:40.1478271Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2025-12-04T09:22:40.1479089Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2025-12-04T09:22:40.1479901Z  exit 1 2025-12-04T09:22:40.1480162Z fi 2025-12-04T09:22:40.1480391Z  2025-12-04T09:22:40.1480811Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2025-12-04T09:22:40.1481588Z # If no image exists but the hash is the same as the previous hash then we should error out here 2025-12-04T09:22:40.1482280Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2025-12-04T09:22:40.1483074Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2025-12-04T09:22:40.1483980Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2025-12-04T09:22:40.1484529Z fi 2025-12-04T09:22:40.1484759Z  2025-12-04T09:22:40.1485053Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2025-12-04T09:22:40.1491197Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:40.1491635Z env: 2025-12-04T09:22:40.1491869Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:40.1492190Z DOCKER_BUILD_DIR: .ci/docker 2025-12-04T09:22:40.1492590Z BASE_REVISION: ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:22:40.1493695Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:40.1495113Z DOCKER_TAG: pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:40.1498768Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:40.1499253Z DOCKER_PUSH: 2025-12-04T09:22:40.1499506Z ##[endgroup] 2025-12-04T09:22:40.1526409Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:40.1526969Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:40.1529859Z + aws ecr get-login-password --region us-east-1 2025-12-04T09:22:40.1530516Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:40.7711981Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-12-04T09:22:40.7712730Z Configure a credential helper to remove this warning. See 2025-12-04T09:22:40.7713419Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-12-04T09:22:40.7713900Z 2025-12-04T09:22:40.7714020Z Login Succeeded 2025-12-04T09:22:40.7730901Z ++ date +%s 2025-12-04T09:22:40.7740285Z + START_TIME=1764840160 2025-12-04T09:22:40.7744686Z ++ date +%s 2025-12-04T09:22:40.7755249Z + [[ 1764832960 -lt 1764840160 ]] 2025-12-04T09:22:40.7756441Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:41.0157219Z { 2025-12-04T09:22:41.0157543Z "schemaVersion": 2, 2025-12-04T09:22:41.0158075Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2025-12-04T09:22:41.0158611Z "config": { 2025-12-04T09:22:41.0159011Z "mediaType": "application/vnd.docker.container.image.v1+json", 2025-12-04T09:22:41.0159497Z "size": 33581, 2025-12-04T09:22:41.0159978Z "digest": "sha256:40bd34edc3497e05e2150804a15313235b6426bcc4b8c19c03c3daf5524be913" 2025-12-04T09:22:41.0160542Z }, 2025-12-04T09:22:41.0160776Z "layers": [ 2025-12-04T09:22:41.0161019Z { 2025-12-04T09:22:41.0161408Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0161915Z "size": 30447951, 2025-12-04T09:22:41.0162432Z "digest": "sha256:63e5bc7682b85ae57a1221210f64d62e7a90b0a30f19af4ca734b8242ae49d63" 2025-12-04T09:22:41.0163273Z }, 2025-12-04T09:22:41.0163502Z { 2025-12-04T09:22:41.0163870Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0164361Z "size": 1554, 2025-12-04T09:22:41.0164856Z "digest": "sha256:6dc15eca51381c13be16385052dd2378ab1dce5fb77f7e7bceab34ed72e6e0e5" 2025-12-04T09:22:41.0165422Z }, 2025-12-04T09:22:41.0165634Z { 2025-12-04T09:22:41.0166013Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0166506Z "size": 313274856, 2025-12-04T09:22:41.0166993Z "digest": "sha256:459f4df18f07c473cb662e8033f9dab9f68c9ac71608767ad055999fae6b31e2" 2025-12-04T09:22:41.0167548Z }, 2025-12-04T09:22:41.0167772Z { 2025-12-04T09:22:41.0168254Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0168729Z "size": 787, 2025-12-04T09:22:41.0169194Z "digest": "sha256:82108541691974f1cc469d0ec89a048191b8de4dc36bf9a0184a4bd50120b69d" 2025-12-04T09:22:41.0169725Z }, 2025-12-04T09:22:41.0169941Z { 2025-12-04T09:22:41.0170313Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0170773Z "size": 106, 2025-12-04T09:22:41.0171232Z "digest": "sha256:3738646b3d92a3f1b6529ba588c00216c306c997ad77730f207c7107c2e5aff3" 2025-12-04T09:22:41.0171768Z }, 2025-12-04T09:22:41.0171987Z { 2025-12-04T09:22:41.0172542Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0173035Z "size": 704, 2025-12-04T09:22:41.0173496Z "digest": "sha256:aa9a10a37b3945f69568695fa4a9321333654a69dd44c55eff4e26bf9bffcad1" 2025-12-04T09:22:41.0174022Z }, 2025-12-04T09:22:41.0174239Z { 2025-12-04T09:22:41.0174607Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0175070Z "size": 1218, 2025-12-04T09:22:41.0175536Z "digest": "sha256:55bac08f3e18897f61d3e7f28ada6547b05b3c36f559a882656834d43de61478" 2025-12-04T09:22:41.0176077Z }, 2025-12-04T09:22:41.0176278Z { 2025-12-04T09:22:41.0176654Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0177128Z "size": 485, 2025-12-04T09:22:41.0177587Z "digest": "sha256:acf6468f6aadc3613e3b8fe73c016ce8d6f47f01e0cfee3a1015d0335c24d691" 2025-12-04T09:22:41.0178139Z }, 2025-12-04T09:22:41.0178459Z + exit 0 2025-12-04T09:22:41.0178691Z { 2025-12-04T09:22:41.0179047Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0179529Z "size": 110362453, 2025-12-04T09:22:41.0180015Z "digest": "sha256:c0c31a0e69b798875a8d2c05d01a6861b2fe623e1970bf7faf822e7df6691d2c" 2025-12-04T09:22:41.0180549Z }, 2025-12-04T09:22:41.0180773Z { 2025-12-04T09:22:41.0181148Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0181611Z "size": 4961, 2025-12-04T09:22:41.0182079Z "digest": "sha256:0f7d0dc70d2d944cc1c8a592f051d3f28d2b062c79166713fb7cc9225b4c97c0" 2025-12-04T09:22:41.0189195Z }, 2025-12-04T09:22:41.0189463Z { 2025-12-04T09:22:41.0189874Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0190370Z "size": 1756, 2025-12-04T09:22:41.0190868Z "digest": "sha256:58c4d15d3bc18a789eb0d1a13f68df0b1d6cea66697fe63adf11b53fec74ba18" 2025-12-04T09:22:41.0191425Z }, 2025-12-04T09:22:41.0191646Z { 2025-12-04T09:22:41.0192027Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0192507Z "size": 724, 2025-12-04T09:22:41.0192997Z "digest": "sha256:2f1cc47b61e1ebefa16f8ee12c5a1df77b109a84a8e8f425e7d65ec51cafdb93" 2025-12-04T09:22:41.0193567Z }, 2025-12-04T09:22:41.0193777Z { 2025-12-04T09:22:41.0194158Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0194649Z "size": 543, 2025-12-04T09:22:41.0210300Z "digest": "sha256:d2472a74103d877e6532d6409a2ac9d54db6672b5d3002be011fdbbd8d360c49" 2025-12-04T09:22:41.0210996Z }, 2025-12-04T09:22:41.0211233Z { 2025-12-04T09:22:41.0211634Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0212328Z "size": 3353192867, 2025-12-04T09:22:41.0212824Z "digest": "sha256:d1fba72936888ea8f8e4c7ad015bfcf3285017b95f3d72795d5b6a0b0ded36fc" 2025-12-04T09:22:41.0213493Z }, 2025-12-04T09:22:41.0213715Z { 2025-12-04T09:22:41.0214072Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0214553Z "size": 32, 2025-12-04T09:22:41.0215021Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0215571Z }, 2025-12-04T09:22:41.0215775Z { 2025-12-04T09:22:41.0216251Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0216737Z "size": 397, 2025-12-04T09:22:41.0217204Z "digest": "sha256:6a9b03ce41a77be16366fba2c6ca0aabefe42ee4879ab5d088b312970e830770" 2025-12-04T09:22:41.0217762Z }, 2025-12-04T09:22:41.0217975Z { 2025-12-04T09:22:41.0218344Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0218825Z "size": 237420, 2025-12-04T09:22:41.0219294Z "digest": "sha256:3519ac15be7978a14201f5d431b6472ed30382cde8628c6401c9db98ccd1a871" 2025-12-04T09:22:41.0219820Z }, 2025-12-04T09:22:41.0220036Z { 2025-12-04T09:22:41.0220407Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0220884Z "size": 231, 2025-12-04T09:22:41.0221339Z "digest": "sha256:dabb51b819a06823e5a1c8d4f9e29b689f38b19ee1e90da40075bd1593b8d0fb" 2025-12-04T09:22:41.0221884Z }, 2025-12-04T09:22:41.0222098Z { 2025-12-04T09:22:41.0222451Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0222928Z "size": 2967944, 2025-12-04T09:22:41.0223407Z "digest": "sha256:8f36833a24d02d47bf18bdc573adbb45afb8f5f06886da9bd671a1a33e3007bd" 2025-12-04T09:22:41.0223943Z }, 2025-12-04T09:22:41.0224158Z { 2025-12-04T09:22:41.0224529Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0224993Z "size": 1472, 2025-12-04T09:22:41.0225476Z "digest": "sha256:ab53c5a853a52de13f11fbb026aaf469754adbe52b02e96f4fbdc05916585244" 2025-12-04T09:22:41.0226020Z }, 2025-12-04T09:22:41.0226220Z { 2025-12-04T09:22:41.0226587Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0227358Z "size": 482, 2025-12-04T09:22:41.0227906Z "digest": "sha256:025a0e5e6ac19cb2d27c9423a5d019a34ea7d4004d426d6b890693ccfe9f54d3" 2025-12-04T09:22:41.0228450Z }, 2025-12-04T09:22:41.0228674Z { 2025-12-04T09:22:41.0229056Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0229538Z "size": 200, 2025-12-04T09:22:41.0230026Z "digest": "sha256:ce3394c8f2109f4538f6e7f2c39f685d0ba028aacac33ddacbb8ef9fd349702e" 2025-12-04T09:22:41.0230589Z }, 2025-12-04T09:22:41.0230798Z { 2025-12-04T09:22:41.0231178Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0231809Z "size": 608, 2025-12-04T09:22:41.0232279Z "digest": "sha256:a5c3888c3a0c5812efbfe307d18de826363900fa8d8097c1ee84a72630aa067b" 2025-12-04T09:22:41.0232839Z }, 2025-12-04T09:22:41.0233061Z { 2025-12-04T09:22:41.0233427Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0233918Z "size": 225, 2025-12-04T09:22:41.0234397Z "digest": "sha256:ed902a3a4e3b916aa869b612baf9cd36745b671b6781d19e3b4c3fadc2f513f5" 2025-12-04T09:22:41.0234958Z }, 2025-12-04T09:22:41.0235165Z { 2025-12-04T09:22:41.0235545Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0236036Z "size": 829, 2025-12-04T09:22:41.0236506Z "digest": "sha256:b4e1efca22beb475459a14f4a571a2512c2b1ed4b24e66bd3f82f1681dc5b1bc" 2025-12-04T09:22:41.0237073Z }, 2025-12-04T09:22:41.0237293Z { 2025-12-04T09:22:41.0237660Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0238155Z "size": 32, 2025-12-04T09:22:41.0238640Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0239377Z }, 2025-12-04T09:22:41.0239699Z { 2025-12-04T09:22:41.0240058Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0240519Z "size": 104, 2025-12-04T09:22:41.0240960Z "digest": "sha256:79fcfd297d9fcfe2810595fc2c6ba503293b59046bc192bb0620a64d9bdff778" 2025-12-04T09:22:41.0241491Z }, 2025-12-04T09:22:41.0241705Z { 2025-12-04T09:22:41.0242050Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0242514Z "size": 1495, 2025-12-04T09:22:41.0242965Z "digest": "sha256:7bd89c134b49f83cfb0725df3615556b60af380b932186c5abd001232b4597cf" 2025-12-04T09:22:41.0243476Z }, 2025-12-04T09:22:41.0243693Z { 2025-12-04T09:22:41.0244057Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0244514Z "size": 458790783, 2025-12-04T09:22:41.0245392Z "digest": "sha256:b7d40b4fd1b9375cfd45d5c593ec0cccc21c80d667ea79f4effb09f04b0705b0" 2025-12-04T09:22:41.0246242Z }, 2025-12-04T09:22:41.0246452Z { 2025-12-04T09:22:41.0246836Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0247327Z "size": 163, 2025-12-04T09:22:41.0247815Z "digest": "sha256:dd1cdd87320d29d4ca09686ab00b76a396efcc6f3ea6d0bfd1f7922e46336ca6" 2025-12-04T09:22:41.0248369Z }, 2025-12-04T09:22:41.0248589Z { 2025-12-04T09:22:41.0248967Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0249448Z "size": 347, 2025-12-04T09:22:41.0249934Z "digest": "sha256:c21a6e1cd03a718ebfb927ab114ebbbee0b57a93908d0265d24b02cf34c0fee3" 2025-12-04T09:22:41.0250507Z }, 2025-12-04T09:22:41.0250714Z { 2025-12-04T09:22:41.0251099Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0251590Z "size": 32, 2025-12-04T09:22:41.0252063Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0252630Z }, 2025-12-04T09:22:41.0252862Z { 2025-12-04T09:22:41.0253225Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0253711Z "size": 106, 2025-12-04T09:22:41.0254194Z "digest": "sha256:0aa3ea0b275426eac8eff6b26ba17c168ae70d1baa5a7845e61d41bd01a9ff43" 2025-12-04T09:22:41.0254763Z }, 2025-12-04T09:22:41.0254970Z { 2025-12-04T09:22:41.0255344Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0255827Z "size": 426, 2025-12-04T09:22:41.0256301Z "digest": "sha256:74e0bdbb05d31425d7a2f40ff0cb2423abaf90f1ed56c4b69f43e8d92c569379" 2025-12-04T09:22:41.0256869Z }, 2025-12-04T09:22:41.0257091Z { 2025-12-04T09:22:41.0257458Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0258056Z "size": 19309400, 2025-12-04T09:22:41.0258543Z "digest": "sha256:ba3aee7dcf03efbd1e7b1611c8f65f385c94b2572d15c099a1535f27b72509f7" 2025-12-04T09:22:41.0259180Z }, 2025-12-04T09:22:41.0259397Z { 2025-12-04T09:22:41.0259773Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0260251Z "size": 108, 2025-12-04T09:22:41.0260700Z "digest": "sha256:8c8d59b8759d40399b07b019a46fc05b9ccabe2b9276f49eea203fd6908ff334" 2025-12-04T09:22:41.0261242Z }, 2025-12-04T09:22:41.0261457Z { 2025-12-04T09:22:41.0261815Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0262290Z "size": 827, 2025-12-04T09:22:41.0262753Z "digest": "sha256:3c960865867b66327e316bf682adabd1867dd9a41cc0aee1507ee8e35fa614a9" 2025-12-04T09:22:41.0263282Z }, 2025-12-04T09:22:41.0263502Z { 2025-12-04T09:22:41.0263869Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0264333Z "size": 724, 2025-12-04T09:22:41.0264807Z "digest": "sha256:2f1cc47b61e1ebefa16f8ee12c5a1df77b109a84a8e8f425e7d65ec51cafdb93" 2025-12-04T09:22:41.0265362Z }, 2025-12-04T09:22:41.0265567Z { 2025-12-04T09:22:41.0265938Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0266512Z "size": 149, 2025-12-04T09:22:41.0267239Z "digest": "sha256:074db79e3832af094ddad6e5c5649347b62e8ba0365229feced3ed0263a0c611" 2025-12-04T09:22:41.0267791Z }, 2025-12-04T09:22:41.0268050Z { 2025-12-04T09:22:41.0268431Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0268906Z "size": 137, 2025-12-04T09:22:41.0269383Z "digest": "sha256:226748f4ff23ba2aef33619c3a639fb44865d5f4515c4b45d4196f4e0e77b187" 2025-12-04T09:22:41.0269942Z }, 2025-12-04T09:22:41.0270148Z { 2025-12-04T09:22:41.0270526Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0271016Z "size": 140, 2025-12-04T09:22:41.0271473Z "digest": "sha256:75d817336f1df7579f5b4be29625c53d594cb78c776f5454cc69438bb1bb0dc9" 2025-12-04T09:22:41.0272028Z }, 2025-12-04T09:22:41.0272248Z { 2025-12-04T09:22:41.0272626Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0273122Z "size": 18890247472, 2025-12-04T09:22:41.0273616Z "digest": "sha256:850870a17b94a4d7c5e55b287cc0b1ddd0982845e11f055765dc358277cdfc54" 2025-12-04T09:22:41.0274167Z }, 2025-12-04T09:22:41.0274374Z { 2025-12-04T09:22:41.0274753Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0275239Z "size": 222, 2025-12-04T09:22:41.0275695Z "digest": "sha256:e6225129924f0a66acaf35efd792901351da58116629962669d9c3acaadbab6e" 2025-12-04T09:22:41.0276246Z }, 2025-12-04T09:22:41.0276465Z { 2025-12-04T09:22:41.0276831Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0277317Z "size": 255, 2025-12-04T09:22:41.0277809Z "digest": "sha256:5cd6b9ec5c3c4a1cf2e2e7a1794aaae173eb5a291b9899bb733e5f36d866fab0" 2025-12-04T09:22:41.0278363Z }, 2025-12-04T09:22:41.0278585Z { 2025-12-04T09:22:41.0279078Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0279665Z "size": 32, 2025-12-04T09:22:41.0280109Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0280650Z }, 2025-12-04T09:22:41.0280860Z { 2025-12-04T09:22:41.0281226Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0281675Z "size": 106, 2025-12-04T09:22:41.0282132Z "digest": "sha256:613d2b1f3db875d1d695081b44aefbf94eeb7815d794b28dd17ec623d3f9dad5" 2025-12-04T09:22:41.0282663Z }, 2025-12-04T09:22:41.0282862Z { 2025-12-04T09:22:41.0283220Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0283684Z "size": 312293443, 2025-12-04T09:22:41.0284152Z "digest": "sha256:388169fffe8a06396f4e8d6fc51d07a83e08f04836995ae28c2e13960cb89fab" 2025-12-04T09:22:41.0284667Z }, 2025-12-04T09:22:41.0284880Z { 2025-12-04T09:22:41.0285244Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0285785Z "size": 6628443345, 2025-12-04T09:22:41.0286269Z "digest": "sha256:6ca498e78b7b821a815bc7c14f057d52f7d9ce8dda8bca0945e6f522c7d208e1" 2025-12-04T09:22:41.0286807Z }, 2025-12-04T09:22:41.0287008Z { 2025-12-04T09:22:41.0287377Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0287841Z "size": 129, 2025-12-04T09:22:41.0288283Z "digest": "sha256:952b753ec7be39e1967f95afb87a880712b6425610adcda2afe3b2211cc8de12" 2025-12-04T09:22:41.0288814Z }, 2025-12-04T09:22:41.0289024Z { 2025-12-04T09:22:41.0289375Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0289845Z "size": 880, 2025-12-04T09:22:41.0290300Z "digest": "sha256:5b6f6de4bdd1c96aa8704c326fb27fa7465ebf388f0748ee56d9f521bb6b6697" 2025-12-04T09:22:41.0290838Z }, 2025-12-04T09:22:41.0291036Z { 2025-12-04T09:22:41.0291398Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0291854Z "size": 724, 2025-12-04T09:22:41.0292301Z "digest": "sha256:2f1cc47b61e1ebefa16f8ee12c5a1df77b109a84a8e8f425e7d65ec51cafdb93" 2025-12-04T09:22:41.0292894Z }, 2025-12-04T09:22:41.0293108Z { 2025-12-04T09:22:41.0293455Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0293915Z "size": 139, 2025-12-04T09:22:41.0294377Z "digest": "sha256:fa23d9952f3cea91af7b5ffce93e64d90d98aefb838aa8a1fe4a6b40cd0eab91" 2025-12-04T09:22:41.0294906Z }, 2025-12-04T09:22:41.0295122Z { 2025-12-04T09:22:41.0295481Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0295946Z "size": 32, 2025-12-04T09:22:41.0296387Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0296912Z }, 2025-12-04T09:22:41.0297114Z { 2025-12-04T09:22:41.0297459Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0297921Z "size": 161, 2025-12-04T09:22:41.0298396Z "digest": "sha256:c2aa9ea4d09ac3edb41e48cdc892fafe72a5cddc98ebdf67d978c6f8d63cd7d2" 2025-12-04T09:22:41.0298934Z }, 2025-12-04T09:22:41.0299141Z { 2025-12-04T09:22:41.0299495Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0299942Z "size": 1011, 2025-12-04T09:22:41.0300385Z "digest": "sha256:a9bbc9e426d367e15c7f0c1faae6417526e219fa8188ae17b9e6c2c3b8083bdc" 2025-12-04T09:22:41.0301494Z }, 2025-12-04T09:22:41.0301715Z { 2025-12-04T09:22:41.0302215Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0302712Z "size": 724, 2025-12-04T09:22:41.0303201Z "digest": "sha256:2f1cc47b61e1ebefa16f8ee12c5a1df77b109a84a8e8f425e7d65ec51cafdb93" 2025-12-04T09:22:41.0303755Z }, 2025-12-04T09:22:41.0303973Z { 2025-12-04T09:22:41.0304349Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0304823Z "size": 135, 2025-12-04T09:22:41.0305306Z "digest": "sha256:82f2371f6dc2c2797baa60e672fd78bb04909cd0248c52cae058a7fd8f215a4f" 2025-12-04T09:22:41.0305876Z }, 2025-12-04T09:22:41.0306085Z { 2025-12-04T09:22:41.0306479Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0307048Z "size": 32, 2025-12-04T09:22:41.0307616Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0308239Z }, 2025-12-04T09:22:41.0308464Z { 2025-12-04T09:22:41.0308831Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0309321Z "size": 158, 2025-12-04T09:22:41.0309805Z "digest": "sha256:c87301921afac5ba5745add04b44f57bf31c551bf48a2574d4424dab64c25ce2" 2025-12-04T09:22:41.0310372Z }, 2025-12-04T09:22:41.0310580Z { 2025-12-04T09:22:41.0310960Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0311446Z "size": 603, 2025-12-04T09:22:41.0311922Z "digest": "sha256:384d566a822f4defa0ecf9c7d37b6d7a5fc5eaab415fe1d3cc02b190ddc19e71" 2025-12-04T09:22:41.0312490Z }, 2025-12-04T09:22:41.0312857Z { 2025-12-04T09:22:41.0313217Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0313696Z "size": 724, 2025-12-04T09:22:41.0314174Z "digest": "sha256:2f1cc47b61e1ebefa16f8ee12c5a1df77b109a84a8e8f425e7d65ec51cafdb93" 2025-12-04T09:22:41.0314717Z }, 2025-12-04T09:22:41.0314921Z { 2025-12-04T09:22:41.0315282Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0315747Z "size": 155, 2025-12-04T09:22:41.0316214Z "digest": "sha256:32ddcdd5dd6920acb12ca95671a164346308ec7743afbe8a333142733c42bafd" 2025-12-04T09:22:41.0316757Z }, 2025-12-04T09:22:41.0316961Z { 2025-12-04T09:22:41.0317318Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0317789Z "size": 32, 2025-12-04T09:22:41.0318252Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0318793Z }, 2025-12-04T09:22:41.0318998Z { 2025-12-04T09:22:41.0319361Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0320028Z "size": 188, 2025-12-04T09:22:41.0320482Z "digest": "sha256:a116a32ceaa15bbd55b0f4281fb0040de221c62e29659b23dd71ecd2382a969a" 2025-12-04T09:22:41.0321017Z }, 2025-12-04T09:22:41.0321211Z { 2025-12-04T09:22:41.0321567Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0322030Z "size": 1371, 2025-12-04T09:22:41.0322492Z "digest": "sha256:f6fbbbe6067e383cadb530c3f53b113e9826843fb2348ea1f9a000a4790fa822" 2025-12-04T09:22:41.0323016Z }, 2025-12-04T09:22:41.0323218Z { 2025-12-04T09:22:41.0323571Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0324024Z "size": 32, 2025-12-04T09:22:41.0324480Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0325014Z }, 2025-12-04T09:22:41.0325209Z { 2025-12-04T09:22:41.0325568Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0326038Z "size": 137, 2025-12-04T09:22:41.0326480Z "digest": "sha256:648018658875ac3356b277abe3d98df9a8b7a0345f131598c07f5e9f9318b740" 2025-12-04T09:22:41.0327002Z }, 2025-12-04T09:22:41.0327212Z { 2025-12-04T09:22:41.0327561Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0328024Z "size": 528, 2025-12-04T09:22:41.0328493Z "digest": "sha256:5e2f7eac20ad5128eebe0ba7dbd08111d28e7f65c26c7fea7cd6dc7a2c0725b9" 2025-12-04T09:22:41.0329036Z }, 2025-12-04T09:22:41.0329342Z { 2025-12-04T09:22:41.0329683Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0330135Z "size": 32, 2025-12-04T09:22:41.0330564Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0331083Z }, 2025-12-04T09:22:41.0331280Z { 2025-12-04T09:22:41.0331618Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0332067Z "size": 104, 2025-12-04T09:22:41.0332501Z "digest": "sha256:0d1243af7593a042ff443492a4cac70f817dc8c56ac45a0c138d102f40c1cc07" 2025-12-04T09:22:41.0333011Z }, 2025-12-04T09:22:41.0333205Z { 2025-12-04T09:22:41.0333549Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0333991Z "size": 435, 2025-12-04T09:22:41.0334434Z "digest": "sha256:1f6d5b941ea1f6f07c190f5ca1d4a5172e4d696ad597ae58dcd37b297f60b556" 2025-12-04T09:22:41.0334955Z }, 2025-12-04T09:22:41.0335152Z { 2025-12-04T09:22:41.0335490Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0335934Z "size": 32, 2025-12-04T09:22:41.0336368Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0336876Z }, 2025-12-04T09:22:41.0337067Z { 2025-12-04T09:22:41.0337408Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0337849Z "size": 107, 2025-12-04T09:22:41.0338364Z "digest": "sha256:4444f70d73cefec9f161242e9def76078af5abc7a3ad3e5d6e7d4f1efed07939" 2025-12-04T09:22:41.0338892Z }, 2025-12-04T09:22:41.0339080Z { 2025-12-04T09:22:41.0339426Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0339875Z "size": 1896, 2025-12-04T09:22:41.0340309Z "digest": "sha256:bf45550ddbad2524852930466f963e432429f2b86f7f6ccf0415fc56980004ce" 2025-12-04T09:22:41.0340809Z }, 2025-12-04T09:22:41.0341007Z { 2025-12-04T09:22:41.0341351Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0341797Z "size": 245588924, 2025-12-04T09:22:41.0342254Z "digest": "sha256:30fff4f4bad325a3a2d91463ca6702c411392e5b0646a5cd4f47fcae56c55639" 2025-12-04T09:22:41.0342768Z }, 2025-12-04T09:22:41.0342957Z { 2025-12-04T09:22:41.0343308Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0343760Z "size": 106, 2025-12-04T09:22:41.0344199Z "digest": "sha256:cca4db1ba155a3afd1fd645cd3223e9a4de9c785f8559387f21da187672d9e9e" 2025-12-04T09:22:41.0344781Z }, 2025-12-04T09:22:41.0344981Z { 2025-12-04T09:22:41.0345319Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0345782Z "size": 165, 2025-12-04T09:22:41.0346240Z "digest": "sha256:f0ffa379f4eba2f3ddc323f45f226f39aed5a7e93880884a97bd4d6a91864dc1" 2025-12-04T09:22:41.0346877Z }, 2025-12-04T09:22:41.0347245Z { 2025-12-04T09:22:41.0347627Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0348159Z "size": 7942, 2025-12-04T09:22:41.0348629Z "digest": "sha256:df645e678f95db120f96e4c4c66400d315ed9214d9085216b417348cf682d43a" 2025-12-04T09:22:41.0349180Z }, 2025-12-04T09:22:41.0349398Z { 2025-12-04T09:22:41.0349762Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0350251Z "size": 8076, 2025-12-04T09:22:41.0350736Z "digest": "sha256:52ad6c17d10308710fb2dc855f911246b7a369ccf5c3142cb3ad7fb182f708b8" 2025-12-04T09:22:41.0351280Z }, 2025-12-04T09:22:41.0351504Z { 2025-12-04T09:22:41.0351880Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0352358Z "size": 302, 2025-12-04T09:22:41.0352833Z "digest": "sha256:34408374e32c70dbcfd7e0516edaa8698c25e6e3dbcd920978f2b3a801bbe659" 2025-12-04T09:22:41.0353394Z }, 2025-12-04T09:22:41.0353616Z { 2025-12-04T09:22:41.0353983Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0354481Z "size": 32, 2025-12-04T09:22:41.0354961Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0355512Z }, 2025-12-04T09:22:41.0355732Z { 2025-12-04T09:22:41.0356111Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0356587Z "size": 108, 2025-12-04T09:22:41.0357072Z "digest": "sha256:27803b661d9e2cb1ce49c0a100ce824fbcbdd1f3a3ceda4ed5affbcb07ac3be4" 2025-12-04T09:22:41.0357640Z }, 2025-12-04T09:22:41.0357851Z { 2025-12-04T09:22:41.0358227Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0358723Z "size": 54145699, 2025-12-04T09:22:41.0359325Z "digest": "sha256:07dafc893dea952d7677124b42e90b29b167fabe81e8a9d8b3fb4b0aa11b66ec" 2025-12-04T09:22:41.0359867Z }, 2025-12-04T09:22:41.0360082Z { 2025-12-04T09:22:41.0360453Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2025-12-04T09:22:41.0360913Z "size": 32, 2025-12-04T09:22:41.0361379Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2025-12-04T09:22:41.0361929Z } 2025-12-04T09:22:41.0362179Z ] 2025-12-04T09:22:41.0362401Z } 2025-12-04T09:22:41.0399006Z ##[group]Run set -eux 2025-12-04T09:22:41.0399326Z set -eux 2025-12-04T09:22:41.0399793Z # It's ok if this steps fails, it would then be an anonymous user like what we used to have 2025-12-04T09:22:41.0402300Z aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token | jq --raw-output '.SecretString' | jq -r .docker_hub_readonly_token | docker login --username pytorchbot --password-stdin || true 2025-12-04T09:22:41.0409704Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:41.0410159Z env: 2025-12-04T09:22:41.0410407Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:41.0410716Z ##[endgroup] 2025-12-04T09:22:41.0441563Z + aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token 2025-12-04T09:22:41.0442310Z + jq --raw-output .SecretString 2025-12-04T09:22:41.0443518Z + jq -r .docker_hub_readonly_token 2025-12-04T09:22:41.0444388Z + docker login --username pytorchbot --password-stdin 2025-12-04T09:22:41.6819838Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-12-04T09:22:41.6820591Z Configure a credential helper to remove this warning. See 2025-12-04T09:22:41.6821375Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-12-04T09:22:41.6821870Z 2025-12-04T09:22:41.6822006Z Login Succeeded 2025-12-04T09:22:41.6912553Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*:} 2025-12-04T09:22:41.6912997Z tag=${ECR_DOCKER_IMAGE##*:} 2025-12-04T09:22:41.6913470Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2025-12-04T09:22:41.6920314Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:41.6920750Z env: 2025-12-04T09:22:41.6921000Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:41.6922020Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:41.6923057Z ##[endgroup] 2025-12-04T09:22:41.6952851Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:41.7005132Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2025-12-04T09:22:41.7005646Z with: 2025-12-04T09:22:41.7006633Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:41.7007848Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:41.7008312Z env: 2025-12-04T09:22:41.7008550Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:41.7008863Z ##[endgroup] 2025-12-04T09:22:41.7025736Z ##[group]Run set -x 2025-12-04T09:22:41.7026044Z set -x 2025-12-04T09:22:41.7026291Z set +e 2025-12-04T09:22:41.7026546Z  2025-12-04T09:22:41.7026921Z login() { 2025-12-04T09:22:41.7040470Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2025-12-04T09:22:41.7041180Z } 2025-12-04T09:22:41.7041420Z  2025-12-04T09:22:41.7041672Z retry () { 2025-12-04T09:22:41.7041967Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2025-12-04T09:22:41.7042332Z } 2025-12-04T09:22:41.7042568Z  2025-12-04T09:22:41.7042840Z retry login "${DOCKER_REGISTRY}" 2025-12-04T09:22:41.7043179Z  2025-12-04T09:22:41.7043746Z IMAGE_SIZE=$(docker manifest inspect "${DOCKER_IMAGE}" | jq '[.layers[].size, .config.size] | add / 1024 / 1024') 2025-12-04T09:22:41.7044519Z echo "Compressed size of image in MB: ${IMAGE_SIZE}" 2025-12-04T09:22:41.7044938Z  2025-12-04T09:22:41.7045173Z set -e 2025-12-04T09:22:41.7045566Z # ignore output since only exit code is used for conditional 2025-12-04T09:22:41.7046137Z # only pull docker image if it's not available locally 2025-12-04T09:22:41.7046755Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2025-12-04T09:22:41.7047345Z  retry docker pull "${DOCKER_IMAGE}" 2025-12-04T09:22:41.7047717Z fi 2025-12-04T09:22:41.7053803Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:22:41.7054232Z env: 2025-12-04T09:22:41.7054474Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:22:41.7055713Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:41.7057332Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:41.7057802Z ##[endgroup] 2025-12-04T09:22:41.7083841Z + set +e 2025-12-04T09:22:41.7084502Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:41.7085083Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:41.7088717Z + aws ecr get-login-password --region us-east-1 2025-12-04T09:22:41.7089388Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2025-12-04T09:22:42.3066573Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2025-12-04T09:22:42.3067502Z Configure a credential helper to remove this warning. See 2025-12-04T09:22:42.3068184Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2025-12-04T09:22:42.3068942Z 2025-12-04T09:22:42.3069068Z Login Succeeded 2025-12-04T09:22:42.3089841Z ++ docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:42.3091057Z ++ jq '[.layers[].size, .config.size] | add / 1024 / 1024' 2025-12-04T09:22:42.4949560Z + IMAGE_SIZE=29010.187264442444 2025-12-04T09:22:42.4950079Z + echo 'Compressed size of image in MB: 29010.187264442444' 2025-12-04T09:22:42.4950543Z + set -e 2025-12-04T09:22:42.4951929Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:42.4953149Z Compressed size of image in MB: 29010.187264442444 2025-12-04T09:22:42.5097146Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:42.5098978Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:22:42.7552223Z pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a: Pulling from pytorch/ci-image 2025-12-04T09:22:42.7553998Z 63e5bc7682b8: Pulling fs layer 2025-12-04T09:22:42.7554534Z 6dc15eca5138: Pulling fs layer 2025-12-04T09:22:42.7554995Z 459f4df18f07: Pulling fs layer 2025-12-04T09:22:42.7555328Z 821085416919: Pulling fs layer 2025-12-04T09:22:42.7555643Z 3738646b3d92: Pulling fs layer 2025-12-04T09:22:42.7555978Z aa9a10a37b39: Pulling fs layer 2025-12-04T09:22:42.7556348Z 55bac08f3e18: Pulling fs layer 2025-12-04T09:22:42.7556672Z acf6468f6aad: Pulling fs layer 2025-12-04T09:22:42.7557008Z c0c31a0e69b7: Pulling fs layer 2025-12-04T09:22:42.7557348Z 0f7d0dc70d2d: Pulling fs layer 2025-12-04T09:22:42.7557704Z 58c4d15d3bc1: Pulling fs layer 2025-12-04T09:22:42.7558012Z aa9a10a37b39: Waiting 2025-12-04T09:22:42.7558301Z 55bac08f3e18: Waiting 2025-12-04T09:22:42.7558602Z 2f1cc47b61e1: Pulling fs layer 2025-12-04T09:22:42.7558921Z d2472a74103d: Pulling fs layer 2025-12-04T09:22:42.7559350Z 3738646b3d92: Waiting 2025-12-04T09:22:42.7559843Z d1fba7293688: Pulling fs layer 2025-12-04T09:22:42.7560442Z 4f4fb700ef54: Pulling fs layer 2025-12-04T09:22:42.7561029Z 2f1cc47b61e1: Waiting 2025-12-04T09:22:42.7561547Z d2472a74103d: Waiting 2025-12-04T09:22:42.7562038Z 821085416919: Waiting 2025-12-04T09:22:42.7562536Z 0f7d0dc70d2d: Waiting 2025-12-04T09:22:42.7563074Z 6a9b03ce41a7: Pulling fs layer 2025-12-04T09:22:42.7563670Z 3519ac15be79: Pulling fs layer 2025-12-04T09:22:42.7564314Z dabb51b819a0: Pulling fs layer 2025-12-04T09:22:42.7564913Z 8f36833a24d0: Pulling fs layer 2025-12-04T09:22:42.7565537Z d1fba7293688: Waiting 2025-12-04T09:22:42.7566037Z 58c4d15d3bc1: Waiting 2025-12-04T09:22:42.7566569Z 4f4fb700ef54: Waiting 2025-12-04T09:22:42.7567118Z ab53c5a853a5: Pulling fs layer 2025-12-04T09:22:42.7567697Z 8f36833a24d0: Waiting 2025-12-04T09:22:42.7568200Z 025a0e5e6ac1: Pulling fs layer 2025-12-04T09:22:42.7568694Z ab53c5a853a5: Waiting 2025-12-04T09:22:42.7569187Z ce3394c8f210: Pulling fs layer 2025-12-04T09:22:42.7569814Z a5c3888c3a0c: Pulling fs layer 2025-12-04T09:22:42.7570255Z 025a0e5e6ac1: Waiting 2025-12-04T09:22:42.7570542Z ed902a3a4e3b: Pulling fs layer 2025-12-04T09:22:42.7570857Z ce3394c8f210: Waiting 2025-12-04T09:22:42.7571133Z a5c3888c3a0c: Waiting 2025-12-04T09:22:42.7571480Z b4e1efca22be: Pulling fs layer 2025-12-04T09:22:42.7571991Z 6a9b03ce41a7: Waiting 2025-12-04T09:22:42.7572323Z c0c31a0e69b7: Waiting 2025-12-04T09:22:42.7572608Z ed902a3a4e3b: Waiting 2025-12-04T09:22:42.7572915Z 79fcfd297d9f: Pulling fs layer 2025-12-04T09:22:42.7573240Z b4e1efca22be: Waiting 2025-12-04T09:22:42.7573523Z acf6468f6aad: Waiting 2025-12-04T09:22:42.7573806Z 7bd89c134b49: Pulling fs layer 2025-12-04T09:22:42.7574395Z b7d40b4fd1b9: Pulling fs layer 2025-12-04T09:22:42.7574729Z dd1cdd87320d: Pulling fs layer 2025-12-04T09:22:42.7575049Z c21a6e1cd03a: Pulling fs layer 2025-12-04T09:22:42.7575369Z 7bd89c134b49: Waiting 2025-12-04T09:22:42.7575653Z 79fcfd297d9f: Waiting 2025-12-04T09:22:42.7576030Z 0aa3ea0b2754: Pulling fs layer 2025-12-04T09:22:42.7576638Z 74e0bdbb05d3: Pulling fs layer 2025-12-04T09:22:42.7576982Z ba3aee7dcf03: Pulling fs layer 2025-12-04T09:22:42.7577296Z dd1cdd87320d: Waiting 2025-12-04T09:22:42.7577742Z c21a6e1cd03a: Waiting 2025-12-04T09:22:42.7578105Z 0aa3ea0b2754: Waiting 2025-12-04T09:22:42.7578389Z 8c8d59b8759d: Pulling fs layer 2025-12-04T09:22:42.7578868Z 3c960865867b: Pulling fs layer 2025-12-04T09:22:42.7579305Z 074db79e3832: Pulling fs layer 2025-12-04T09:22:42.7580006Z 8c8d59b8759d: Waiting 2025-12-04T09:22:42.7580317Z 226748f4ff23: Pulling fs layer 2025-12-04T09:22:42.7580639Z ba3aee7dcf03: Waiting 2025-12-04T09:22:42.7580937Z 75d817336f1d: Pulling fs layer 2025-12-04T09:22:42.7581248Z 3c960865867b: Waiting 2025-12-04T09:22:42.7581537Z 850870a17b94: Pulling fs layer 2025-12-04T09:22:42.7581862Z e6225129924f: Pulling fs layer 2025-12-04T09:22:42.7582161Z 074db79e3832: Waiting 2025-12-04T09:22:42.7582438Z 226748f4ff23: Waiting 2025-12-04T09:22:42.7582710Z 75d817336f1d: Waiting 2025-12-04T09:22:42.7582988Z 5cd6b9ec5c3c: Pulling fs layer 2025-12-04T09:22:42.7583324Z 850870a17b94: Waiting 2025-12-04T09:22:42.7583601Z 613d2b1f3db8: Pulling fs layer 2025-12-04T09:22:42.7583920Z e6225129924f: Waiting 2025-12-04T09:22:42.7584209Z 388169fffe8a: Pulling fs layer 2025-12-04T09:22:42.7584517Z 5cd6b9ec5c3c: Waiting 2025-12-04T09:22:42.7584971Z 6ca498e78b7b: Pulling fs layer 2025-12-04T09:22:42.7585452Z 613d2b1f3db8: Waiting 2025-12-04T09:22:42.7585740Z 952b753ec7be: Pulling fs layer 2025-12-04T09:22:42.7586077Z 5b6f6de4bdd1: Pulling fs layer 2025-12-04T09:22:42.7586402Z 388169fffe8a: Waiting 2025-12-04T09:22:42.7586930Z 74e0bdbb05d3: Waiting 2025-12-04T09:22:42.7587207Z 6ca498e78b7b: Waiting 2025-12-04T09:22:42.7587653Z fa23d9952f3c: Pulling fs layer 2025-12-04T09:22:42.7588204Z 952b753ec7be: Waiting 2025-12-04T09:22:42.7588562Z 5b6f6de4bdd1: Waiting 2025-12-04T09:22:42.7588873Z fa23d9952f3c: Waiting 2025-12-04T09:22:42.7589173Z c2aa9ea4d09a: Pulling fs layer 2025-12-04T09:22:42.7589500Z a9bbc9e426d3: Pulling fs layer 2025-12-04T09:22:42.7589841Z 82f2371f6dc2: Pulling fs layer 2025-12-04T09:22:42.7590175Z c87301921afa: Pulling fs layer 2025-12-04T09:22:42.7590505Z 384d566a822f: Pulling fs layer 2025-12-04T09:22:42.7590813Z c2aa9ea4d09a: Waiting 2025-12-04T09:22:42.7591101Z c87301921afa: Waiting 2025-12-04T09:22:42.7591386Z a9bbc9e426d3: Waiting 2025-12-04T09:22:42.7591665Z 32ddcdd5dd69: Pulling fs layer 2025-12-04T09:22:42.7591985Z 32ddcdd5dd69: Waiting 2025-12-04T09:22:42.7592269Z 82f2371f6dc2: Waiting 2025-12-04T09:22:42.7592528Z 384d566a822f: Waiting 2025-12-04T09:22:42.7592814Z a116a32ceaa1: Pulling fs layer 2025-12-04T09:22:42.7593329Z f6fbbbe6067e: Pulling fs layer 2025-12-04T09:22:42.7593660Z 648018658875: Pulling fs layer 2025-12-04T09:22:42.7593987Z 5e2f7eac20ad: Pulling fs layer 2025-12-04T09:22:42.7594317Z 0d1243af7593: Pulling fs layer 2025-12-04T09:22:42.7594755Z 1f6d5b941ea1: Pulling fs layer 2025-12-04T09:22:42.7595162Z a116a32ceaa1: Waiting 2025-12-04T09:22:42.7595569Z 0d1243af7593: Waiting 2025-12-04T09:22:42.7595969Z 1f6d5b941ea1: Waiting 2025-12-04T09:22:42.7596254Z f6fbbbe6067e: Waiting 2025-12-04T09:22:42.7596531Z 648018658875: Waiting 2025-12-04T09:22:42.7596806Z 4444f70d73ce: Pulling fs layer 2025-12-04T09:22:42.7597128Z 5e2f7eac20ad: Waiting 2025-12-04T09:22:42.7597402Z 4444f70d73ce: Waiting 2025-12-04T09:22:42.7597693Z bf45550ddbad: Pulling fs layer 2025-12-04T09:22:42.7598017Z 30fff4f4bad3: Pulling fs layer 2025-12-04T09:22:42.7598337Z bf45550ddbad: Waiting 2025-12-04T09:22:42.7598640Z cca4db1ba155: Pulling fs layer 2025-12-04T09:22:42.7598963Z f0ffa379f4eb: Pulling fs layer 2025-12-04T09:22:42.7599298Z df645e678f95: Pulling fs layer 2025-12-04T09:22:42.7599627Z 52ad6c17d103: Pulling fs layer 2025-12-04T09:22:42.7600065Z 34408374e32c: Pulling fs layer 2025-12-04T09:22:42.7600395Z 27803b661d9e: Pulling fs layer 2025-12-04T09:22:42.7601152Z 07dafc893dea: Pulling fs layer 2025-12-04T09:22:42.7601510Z 30fff4f4bad3: Waiting 2025-12-04T09:22:42.7601797Z 52ad6c17d103: Waiting 2025-12-04T09:22:42.7602250Z cca4db1ba155: Waiting 2025-12-04T09:22:42.7602701Z 34408374e32c: Waiting 2025-12-04T09:22:42.7603175Z 07dafc893dea: Waiting 2025-12-04T09:22:42.7603632Z df645e678f95: Waiting 2025-12-04T09:22:42.7603903Z f0ffa379f4eb: Waiting 2025-12-04T09:22:42.7604270Z 27803b661d9e: Waiting 2025-12-04T09:22:42.8375677Z 6dc15eca5138: Verifying Checksum 2025-12-04T09:22:42.8376092Z 6dc15eca5138: Download complete 2025-12-04T09:22:42.9225941Z 821085416919: Verifying Checksum 2025-12-04T09:22:42.9226627Z 821085416919: Download complete 2025-12-04T09:22:43.0969665Z aa9a10a37b39: Verifying Checksum 2025-12-04T09:22:43.0970082Z aa9a10a37b39: Download complete 2025-12-04T09:22:43.1192358Z 63e5bc7682b8: Verifying Checksum 2025-12-04T09:22:43.1192814Z 63e5bc7682b8: Download complete 2025-12-04T09:22:43.1932793Z acf6468f6aad: Verifying Checksum 2025-12-04T09:22:43.1933222Z acf6468f6aad: Download complete 2025-12-04T09:22:43.2034642Z 55bac08f3e18: Verifying Checksum 2025-12-04T09:22:43.2035058Z 55bac08f3e18: Download complete 2025-12-04T09:22:43.2839389Z 0f7d0dc70d2d: Verifying Checksum 2025-12-04T09:22:43.2840051Z 0f7d0dc70d2d: Download complete 2025-12-04T09:22:43.3580682Z 58c4d15d3bc1: Verifying Checksum 2025-12-04T09:22:43.3581128Z 58c4d15d3bc1: Download complete 2025-12-04T09:22:43.4480146Z 2f1cc47b61e1: Verifying Checksum 2025-12-04T09:22:43.4480596Z 2f1cc47b61e1: Download complete 2025-12-04T09:22:43.5183982Z d2472a74103d: Download complete 2025-12-04T09:22:44.0704660Z 63e5bc7682b8: Pull complete 2025-12-04T09:22:44.0933924Z 6dc15eca5138: Pull complete 2025-12-04T09:22:44.3622487Z c0c31a0e69b7: Verifying Checksum 2025-12-04T09:22:44.3623112Z c0c31a0e69b7: Download complete 2025-12-04T09:22:44.3697444Z 4f4fb700ef54: Verifying Checksum 2025-12-04T09:22:44.3698202Z 4f4fb700ef54: Download complete 2025-12-04T09:22:44.4706833Z 6a9b03ce41a7: Verifying Checksum 2025-12-04T09:22:44.4707689Z 6a9b03ce41a7: Download complete 2025-12-04T09:22:44.5687964Z 3519ac15be79: Download complete 2025-12-04T09:22:44.6615181Z dabb51b819a0: Download complete 2025-12-04T09:22:44.7722283Z 8f36833a24d0: Download complete 2025-12-04T09:22:44.8587066Z ab53c5a853a5: Verifying Checksum 2025-12-04T09:22:44.8587502Z ab53c5a853a5: Download complete 2025-12-04T09:22:44.9460157Z 025a0e5e6ac1: Verifying Checksum 2025-12-04T09:22:44.9460837Z 025a0e5e6ac1: Download complete 2025-12-04T09:22:45.0294171Z ce3394c8f210: Verifying Checksum 2025-12-04T09:22:45.0294851Z ce3394c8f210: Download complete 2025-12-04T09:22:45.1002010Z a5c3888c3a0c: Verifying Checksum 2025-12-04T09:22:45.1002438Z a5c3888c3a0c: Download complete 2025-12-04T09:22:45.1906987Z ed902a3a4e3b: Verifying Checksum 2025-12-04T09:22:45.1907633Z ed902a3a4e3b: Download complete 2025-12-04T09:22:45.2860113Z b4e1efca22be: Verifying Checksum 2025-12-04T09:22:45.2860593Z b4e1efca22be: Download complete 2025-12-04T09:22:45.3646032Z 79fcfd297d9f: Verifying Checksum 2025-12-04T09:22:45.3646685Z 79fcfd297d9f: Download complete 2025-12-04T09:22:45.4572961Z 7bd89c134b49: Download complete 2025-12-04T09:22:45.9499960Z 459f4df18f07: Download complete 2025-12-04T09:22:46.0214984Z dd1cdd87320d: Verifying Checksum 2025-12-04T09:22:46.0215436Z dd1cdd87320d: Download complete 2025-12-04T09:22:46.1085614Z c21a6e1cd03a: Download complete 2025-12-04T09:22:46.2105472Z 0aa3ea0b2754: Verifying Checksum 2025-12-04T09:22:46.2105912Z 0aa3ea0b2754: Download complete 2025-12-04T09:22:46.2858805Z 74e0bdbb05d3: Verifying Checksum 2025-12-04T09:22:46.2859222Z 74e0bdbb05d3: Download complete 2025-12-04T09:22:46.5290760Z ba3aee7dcf03: Verifying Checksum 2025-12-04T09:22:46.5291207Z ba3aee7dcf03: Download complete 2025-12-04T09:22:46.6176057Z 8c8d59b8759d: Verifying Checksum 2025-12-04T09:22:46.6176490Z 8c8d59b8759d: Download complete 2025-12-04T09:22:46.6769663Z 3c960865867b: Verifying Checksum 2025-12-04T09:22:46.6770186Z 3c960865867b: Download complete 2025-12-04T09:22:46.7323302Z 074db79e3832: Verifying Checksum 2025-12-04T09:22:46.7323745Z 074db79e3832: Download complete 2025-12-04T09:22:46.8047997Z 226748f4ff23: Verifying Checksum 2025-12-04T09:22:46.8048528Z 226748f4ff23: Download complete 2025-12-04T09:22:46.8691987Z 75d817336f1d: Download complete 2025-12-04T09:22:50.0949179Z b7d40b4fd1b9: Verifying Checksum 2025-12-04T09:22:50.0949831Z b7d40b4fd1b9: Download complete 2025-12-04T09:22:50.1836976Z e6225129924f: Verifying Checksum 2025-12-04T09:22:50.1837394Z e6225129924f: Download complete 2025-12-04T09:22:50.2865872Z 5cd6b9ec5c3c: Verifying Checksum 2025-12-04T09:22:50.2866314Z 5cd6b9ec5c3c: Download complete 2025-12-04T09:22:50.3560714Z 613d2b1f3db8: Verifying Checksum 2025-12-04T09:22:50.3561182Z 613d2b1f3db8: Download complete 2025-12-04T09:22:53.5689223Z 388169fffe8a: Verifying Checksum 2025-12-04T09:22:53.5689668Z 388169fffe8a: Download complete 2025-12-04T09:22:54.8154511Z 459f4df18f07: Pull complete 2025-12-04T09:22:54.9888929Z 821085416919: Pull complete 2025-12-04T09:22:55.1739780Z 3738646b3d92: Pull complete 2025-12-04T09:22:55.3443544Z aa9a10a37b39: Pull complete 2025-12-04T09:22:55.5321445Z 55bac08f3e18: Pull complete 2025-12-04T09:22:55.7130512Z acf6468f6aad: Pull complete 2025-12-04T09:22:58.3842066Z c0c31a0e69b7: Pull complete 2025-12-04T09:22:58.5884829Z 0f7d0dc70d2d: Pull complete 2025-12-04T09:22:58.8205279Z 58c4d15d3bc1: Pull complete 2025-12-04T09:22:59.0217329Z 2f1cc47b61e1: Pull complete 2025-12-04T09:22:59.2091648Z d2472a74103d: Pull complete 2025-12-04T09:23:17.1148298Z d1fba7293688: Verifying Checksum 2025-12-04T09:23:17.1148758Z d1fba7293688: Download complete 2025-12-04T09:23:17.2086175Z 952b753ec7be: Download complete 2025-12-04T09:23:17.2796489Z 5b6f6de4bdd1: Download complete 2025-12-04T09:23:17.3408448Z fa23d9952f3c: Verifying Checksum 2025-12-04T09:23:17.3409067Z fa23d9952f3c: Download complete 2025-12-04T09:23:17.4038701Z c2aa9ea4d09a: Verifying Checksum 2025-12-04T09:23:17.4039661Z c2aa9ea4d09a: Download complete 2025-12-04T09:23:17.4666443Z a9bbc9e426d3: Verifying Checksum 2025-12-04T09:23:17.4667006Z a9bbc9e426d3: Download complete 2025-12-04T09:23:17.5440858Z 82f2371f6dc2: Verifying Checksum 2025-12-04T09:23:17.5441585Z 82f2371f6dc2: Download complete 2025-12-04T09:23:17.6059581Z c87301921afa: Verifying Checksum 2025-12-04T09:23:17.6060083Z c87301921afa: Download complete 2025-12-04T09:23:17.6714593Z 384d566a822f: Verifying Checksum 2025-12-04T09:23:17.6715033Z 384d566a822f: Download complete 2025-12-04T09:23:17.7445862Z 32ddcdd5dd69: Download complete 2025-12-04T09:23:17.8711895Z a116a32ceaa1: Verifying Checksum 2025-12-04T09:23:17.9321282Z f6fbbbe6067e: Verifying Checksum 2025-12-04T09:23:17.9321785Z f6fbbbe6067e: Download complete 2025-12-04T09:23:17.9994244Z 648018658875: Verifying Checksum 2025-12-04T09:23:17.9994847Z 648018658875: Download complete 2025-12-04T09:23:18.0620111Z 5e2f7eac20ad: Download complete 2025-12-04T09:23:18.1424867Z 0d1243af7593: Verifying Checksum 2025-12-04T09:23:18.1425348Z 0d1243af7593: Download complete 2025-12-04T09:23:18.2116736Z 1f6d5b941ea1: Verifying Checksum 2025-12-04T09:23:18.2117161Z 1f6d5b941ea1: Download complete 2025-12-04T09:23:18.2958252Z 4444f70d73ce: Download complete 2025-12-04T09:23:18.3958954Z bf45550ddbad: Verifying Checksum 2025-12-04T09:23:18.3959401Z bf45550ddbad: Download complete 2025-12-04T09:23:20.9030854Z 30fff4f4bad3: Verifying Checksum 2025-12-04T09:23:20.9031297Z 30fff4f4bad3: Download complete 2025-12-04T09:23:20.9908171Z cca4db1ba155: Download complete 2025-12-04T09:23:21.0599329Z f0ffa379f4eb: Verifying Checksum 2025-12-04T09:23:21.0600033Z f0ffa379f4eb: Download complete 2025-12-04T09:23:21.1345391Z df645e678f95: Download complete 2025-12-04T09:23:21.2324477Z 52ad6c17d103: Verifying Checksum 2025-12-04T09:23:21.2324936Z 52ad6c17d103: Download complete 2025-12-04T09:23:21.3294849Z 34408374e32c: Download complete 2025-12-04T09:23:21.4059209Z 27803b661d9e: Verifying Checksum 2025-12-04T09:23:21.4059801Z 27803b661d9e: Download complete 2025-12-04T09:23:21.9981724Z 07dafc893dea: Verifying Checksum 2025-12-04T09:23:21.9982213Z 07dafc893dea: Download complete 2025-12-04T09:23:59.9137769Z 6ca498e78b7b: Verifying Checksum 2025-12-04T09:23:59.9138224Z 6ca498e78b7b: Download complete 2025-12-04T09:24:13.7378321Z d1fba7293688: Pull complete 2025-12-04T09:24:13.9355169Z 4f4fb700ef54: Pull complete 2025-12-04T09:24:14.1140749Z 6a9b03ce41a7: Pull complete 2025-12-04T09:24:14.3551285Z 3519ac15be79: Pull complete 2025-12-04T09:24:14.5570659Z dabb51b819a0: Pull complete 2025-12-04T09:24:14.8130792Z 8f36833a24d0: Pull complete 2025-12-04T09:24:14.9984898Z ab53c5a853a5: Pull complete 2025-12-04T09:24:15.2010882Z 025a0e5e6ac1: Pull complete 2025-12-04T09:24:15.4103230Z ce3394c8f210: Pull complete 2025-12-04T09:24:15.6191276Z a5c3888c3a0c: Pull complete 2025-12-04T09:24:15.8226011Z ed902a3a4e3b: Pull complete 2025-12-04T09:24:16.0161433Z b4e1efca22be: Pull complete 2025-12-04T09:24:16.4340910Z 79fcfd297d9f: Pull complete 2025-12-04T09:24:16.6410157Z 7bd89c134b49: Pull complete 2025-12-04T09:24:24.5406737Z b7d40b4fd1b9: Pull complete 2025-12-04T09:24:24.7005869Z dd1cdd87320d: Pull complete 2025-12-04T09:24:24.9023619Z c21a6e1cd03a: Pull complete 2025-12-04T09:24:25.2894381Z 0aa3ea0b2754: Pull complete 2025-12-04T09:24:25.4775067Z 74e0bdbb05d3: Pull complete 2025-12-04T09:24:25.9066513Z ba3aee7dcf03: Pull complete 2025-12-04T09:24:26.1171660Z 8c8d59b8759d: Pull complete 2025-12-04T09:24:26.2855257Z 3c960865867b: Pull complete 2025-12-04T09:24:26.5697868Z 074db79e3832: Pull complete 2025-12-04T09:24:26.6586930Z 226748f4ff23: Pull complete 2025-12-04T09:24:26.6984468Z 75d817336f1d: Pull complete 2025-12-04T09:26:23.5444457Z 850870a17b94: Verifying Checksum 2025-12-04T09:26:23.5444934Z 850870a17b94: Download complete 2025-12-04T09:29:09.1492353Z 850870a17b94: Pull complete 2025-12-04T09:29:09.3259997Z e6225129924f: Pull complete 2025-12-04T09:29:09.5264176Z 5cd6b9ec5c3c: Pull complete 2025-12-04T09:29:09.9555386Z 613d2b1f3db8: Pull complete 2025-12-04T09:29:11.7155993Z 388169fffe8a: Pull complete 2025-12-04T09:30:54.5289381Z 6ca498e78b7b: Pull complete 2025-12-04T09:30:54.7154790Z 952b753ec7be: Pull complete 2025-12-04T09:30:54.9169014Z 5b6f6de4bdd1: Pull complete 2025-12-04T09:30:55.2777112Z fa23d9952f3c: Pull complete 2025-12-04T09:30:55.6510190Z c2aa9ea4d09a: Pull complete 2025-12-04T09:30:55.8272421Z a9bbc9e426d3: Pull complete 2025-12-04T09:30:56.2338431Z 82f2371f6dc2: Pull complete 2025-12-04T09:30:56.6245758Z c87301921afa: Pull complete 2025-12-04T09:30:56.8383908Z 384d566a822f: Pull complete 2025-12-04T09:30:57.2434643Z 32ddcdd5dd69: Pull complete 2025-12-04T09:30:57.6558854Z a116a32ceaa1: Pull complete 2025-12-04T09:30:57.8696495Z f6fbbbe6067e: Pull complete 2025-12-04T09:30:58.3246345Z 648018658875: Pull complete 2025-12-04T09:30:58.5380586Z 5e2f7eac20ad: Pull complete 2025-12-04T09:30:58.9403655Z 0d1243af7593: Pull complete 2025-12-04T09:30:59.1376429Z 1f6d5b941ea1: Pull complete 2025-12-04T09:30:59.5578609Z 4444f70d73ce: Pull complete 2025-12-04T09:30:59.7538478Z bf45550ddbad: Pull complete 2025-12-04T09:31:06.1530319Z 30fff4f4bad3: Pull complete 2025-12-04T09:31:06.3656508Z cca4db1ba155: Pull complete 2025-12-04T09:31:06.5406502Z f0ffa379f4eb: Pull complete 2025-12-04T09:31:06.7326346Z df645e678f95: Pull complete 2025-12-04T09:31:06.9247730Z 52ad6c17d103: Pull complete 2025-12-04T09:31:07.1097024Z 34408374e32c: Pull complete 2025-12-04T09:31:07.5099160Z 27803b661d9e: Pull complete 2025-12-04T09:31:09.5211122Z 07dafc893dea: Pull complete 2025-12-04T09:31:09.8123772Z Digest: sha256:b178ee928adfcab963ebbc6ea05ea1a0f0c605bb095e9076f010bf92d150869d 2025-12-04T09:31:09.8353277Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:31:09.8472842Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:31:09.8536386Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:31:09.8537580Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:31:09.8547875Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:09.8548339Z env: 2025-12-04T09:31:09.8548582Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:09.8548894Z ##[endgroup] 2025-12-04T09:31:09.8785196Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2025-12-04T09:31:09.8785667Z with: 2025-12-04T09:31:09.8785919Z driver-version: 580.82.07 2025-12-04T09:31:09.8786221Z env: 2025-12-04T09:31:09.8786457Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:09.8786875Z ##[endgroup] 2025-12-04T09:31:09.8812695Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:31:09.8813833Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT" 2025-12-04T09:31:09.8820743Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:09.8821188Z env: 2025-12-04T09:31:09.8821440Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:09.8821734Z ##[endgroup] 2025-12-04T09:31:09.8881820Z ##[group]Run set -euo pipefail 2025-12-04T09:31:09.8882194Z set -euo pipefail 2025-12-04T09:31:09.8882558Z  2025-12-04T09:31:09.8882792Z has_gpu=false 2025-12-04T09:31:09.8883083Z devices="" 2025-12-04T09:31:09.8883359Z  2025-12-04T09:31:09.8883669Z if command -v nvidia-smi >/dev/null 2>&1; then 2025-12-04T09:31:09.8884192Z  if nvidia-smi -L >/tmp/nvidia_devices 2>/dev/null; then 2025-12-04T09:31:09.8884648Z  has_gpu=true 2025-12-04T09:31:09.8885002Z  devices=$(cat /tmp/nvidia_devices) 2025-12-04T09:31:09.8885361Z  fi 2025-12-04T09:31:09.8885607Z fi 2025-12-04T09:31:09.8885846Z  2025-12-04T09:31:09.8886095Z if [ "$has_gpu" = false ]; then 2025-12-04T09:31:09.8886548Z  if ls /dev/nvidia* >/tmp/nvidia_devices 2>/dev/null; then 2025-12-04T09:31:09.8886999Z  has_gpu=true 2025-12-04T09:31:09.8887327Z  devices=$(cat /tmp/nvidia_devices) 2025-12-04T09:31:09.8887694Z  fi 2025-12-04T09:31:09.8887938Z fi 2025-12-04T09:31:09.8888166Z  2025-12-04T09:31:09.8888524Z if [ "$has_gpu" = false ] && command -v lspci >/dev/null 2>&1; then 2025-12-04T09:31:09.8889121Z  if lspci | grep -i 'nvidia' >/tmp/nvidia_devices 2>/dev/null; then 2025-12-04T09:31:09.8889607Z  has_gpu=true 2025-12-04T09:31:09.8889934Z  devices=$(cat /tmp/nvidia_devices) 2025-12-04T09:31:09.8890304Z  fi 2025-12-04T09:31:09.8890548Z fi 2025-12-04T09:31:09.8890771Z  2025-12-04T09:31:09.8891120Z printf 'HAS_NVIDIA=%s\n' "$has_gpu" >> "$GITHUB_OUTPUT" 2025-12-04T09:31:09.8891752Z printf 'DETECTED_DEVICES<> "$GITHUB_OUTPUT" 2025-12-04T09:31:09.8897691Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:09.8898128Z env: 2025-12-04T09:31:09.8898377Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:09.8898688Z ##[endgroup] 2025-12-04T09:31:09.9811917Z ##[group]Run if [ "${HAS_NVIDIA}" = "true" ]; then 2025-12-04T09:31:09.9812389Z if [ "${HAS_NVIDIA}" = "true" ]; then 2025-12-04T09:31:09.9812844Z  echo "HAS_NVIDIA_GPU=true" >> "${GITHUB_ENV}" 2025-12-04T09:31:09.9813473Z  echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2025-12-04T09:31:09.9814040Z else 2025-12-04T09:31:09.9814363Z  echo "HAS_NVIDIA_GPU=false" >> "${GITHUB_ENV}" 2025-12-04T09:31:09.9814928Z fi 2025-12-04T09:31:09.9821150Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:09.9821578Z env: 2025-12-04T09:31:09.9821825Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:09.9822128Z HAS_NVIDIA: false 2025-12-04T09:31:09.9822380Z ##[endgroup] 2025-12-04T09:31:09.9917339Z Prepare all required actions 2025-12-04T09:31:09.9964366Z ##[group]Run ./.github/actions/get-workflow-job-id 2025-12-04T09:31:09.9964773Z with: 2025-12-04T09:31:09.9965446Z github-token: *** 2025-12-04T09:31:09.9965723Z env: 2025-12-04T09:31:09.9965960Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:09.9966282Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:09.9966573Z ##[endgroup] 2025-12-04T09:31:10.0007705Z ##[group]Run set -eux 2025-12-04T09:31:10.0008001Z set -eux 2025-12-04T09:31:10.0008533Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2025-12-04T09:31:10.0014705Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:10.0015148Z env: 2025-12-04T09:31:10.0015395Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:10.0015718Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:10.0016193Z GITHUB_TOKEN: *** 2025-12-04T09:31:10.0016451Z ##[endgroup] 2025-12-04T09:31:10.0042050Z + python3 .github/scripts/get_workflow_job_id.py 19923066595 i-01ba3f41dbfc68ae5 2025-12-04T09:31:12.3762551Z Setting output job-id=57118563305 2025-12-04T09:31:12.3763952Z Setting output job-name=periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:31:12.3953323Z ##[group]Run python3 -m pip install psutil==5.9.8 dataclasses_json==0.6.7 nvidia-ml-py==11.525.84 2025-12-04T09:31:12.3954227Z python3 -m pip install psutil==5.9.8 dataclasses_json==0.6.7 nvidia-ml-py==11.525.84 2025-12-04T09:31:12.3955398Z python3 -m tools.stats.monitor --log-interval "$MONITOR_LOG_INTERVAL" --data-collect-interval "$MONITOR_DATA_COLLECT_INTERVAL" > usage_log.txt 2>&1 & 2025-12-04T09:31:12.3956437Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:31:12.3963806Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:12.3964249Z env: 2025-12-04T09:31:12.3964503Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:12.3964799Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:12.3965085Z JOB_ID: 57118563305 2025-12-04T09:31:12.3965769Z JOB_NAME: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:31:12.3966589Z WORKFLOW_NAME: inductor-periodic 2025-12-04T09:31:12.3966940Z WORKFLOW_RUN_ID: 19923066595 2025-12-04T09:31:12.3967252Z MONITOR_LOG_INTERVAL: 5 2025-12-04T09:31:12.3967571Z MONITOR_DATA_COLLECT_INTERVAL: 1 2025-12-04T09:31:12.3967907Z ##[endgroup] 2025-12-04T09:31:13.0971237Z Defaulting to user installation because normal site-packages is not writeable 2025-12-04T09:31:13.6043314Z Collecting psutil==5.9.8 2025-12-04T09:31:13.6236301Z Downloading psutil-5.9.8-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (288 kB) 2025-12-04T09:31:13.7722423Z Collecting dataclasses_json==0.6.7 2025-12-04T09:31:13.7786967Z Downloading dataclasses_json-0.6.7-py3-none-any.whl (28 kB) 2025-12-04T09:31:13.8259983Z Collecting nvidia-ml-py==11.525.84 2025-12-04T09:31:13.8308301Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2025-12-04T09:31:13.8923929Z Collecting typing-inspect<1,>=0.4.0 2025-12-04T09:31:13.8958992Z Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB) 2025-12-04T09:31:14.0583682Z Collecting marshmallow<4.0.0,>=3.18.0 2025-12-04T09:31:14.0618880Z Downloading marshmallow-3.26.1-py3-none-any.whl (50 kB) 2025-12-04T09:31:14.1725665Z Collecting packaging>=17.0 2025-12-04T09:31:14.1763598Z Downloading packaging-25.0-py3-none-any.whl (66 kB) 2025-12-04T09:31:14.2429973Z Collecting mypy-extensions>=0.3.0 2025-12-04T09:31:14.2465408Z Downloading mypy_extensions-1.1.0-py3-none-any.whl (5.0 kB) 2025-12-04T09:31:14.3397487Z Collecting typing-extensions>=3.7.4 2025-12-04T09:31:14.3436709Z Downloading typing_extensions-4.15.0-py3-none-any.whl (44 kB) 2025-12-04T09:31:14.5922542Z Installing collected packages: typing-extensions, packaging, mypy-extensions, typing-inspect, marshmallow, psutil, nvidia-ml-py, dataclasses-json 2025-12-04T09:31:15.1791119Z Successfully installed dataclasses-json-0.6.7 marshmallow-3.26.1 mypy-extensions-1.1.0 nvidia-ml-py-11.525.84 packaging-25.0 psutil-5.9.8 typing-extensions-4.15.0 typing-inspect-0.9.0 2025-12-04T09:31:15.4369003Z Prepare all required actions 2025-12-04T09:31:15.4369505Z Getting action download info 2025-12-04T09:31:15.6474555Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2025-12-04T09:31:16.4252730Z Download action repository 'actions/download-artifact@v4' (SHA:d3f86a106a0bac45b974a628896c90dbdf5c8093) 2025-12-04T09:31:18.2809374Z ##[group]Run ./.github/actions/download-build-artifacts 2025-12-04T09:31:18.2809819Z with: 2025-12-04T09:31:18.2810104Z name: linux-jammy-py3.10-gcc11-build 2025-12-04T09:31:18.2810483Z s3-bucket: gha-artifacts 2025-12-04T09:31:18.2810775Z env: 2025-12-04T09:31:18.2811021Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:18.2811338Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:18.2811633Z ##[endgroup] 2025-12-04T09:31:18.2915595Z ##[group]Run seemethere/download-artifact-s3@v4 2025-12-04T09:31:18.2916003Z with: 2025-12-04T09:31:18.2916289Z name: linux-jammy-py3.10-gcc11-build 2025-12-04T09:31:18.2916872Z s3-bucket: gha-artifacts 2025-12-04T09:31:18.2917233Z region: us-east-1 2025-12-04T09:31:18.2917518Z env: 2025-12-04T09:31:18.2917769Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:18.2918085Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:18.2918364Z ##[endgroup] 2025-12-04T09:31:19.1888370Z (node:57752) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2025-12-04T09:31:19.1888973Z 2025-12-04T09:31:19.1889251Z Please migrate your code to use AWS SDK for JavaScript (v3). 2025-12-04T09:31:19.1889870Z For more information, check the migration guide at https://a.co/7PzMCcy 2025-12-04T09:31:19.1890536Z (Use `node --trace-warnings ...` to show where the warning was created) 2025-12-04T09:31:20.1223298Z Found 1 objects with prefix pytorch/pytorch/19923066595/linux-jammy-py3.10-gcc11-build/ 2025-12-04T09:31:20.1224202Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2025-12-04T09:31:28.9410097Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2025-12-04T09:31:28.9417805Z Artifact download has finished successfully 2025-12-04T09:31:28.9684686Z ##[group]Run unzip -o artifacts.zip 2025-12-04T09:31:28.9685063Z unzip -o artifacts.zip 2025-12-04T09:31:28.9692268Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:28.9692707Z env: 2025-12-04T09:31:28.9692944Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:28.9693285Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:28.9693572Z ##[endgroup] 2025-12-04T09:31:29.0280838Z Archive: artifacts.zip 2025-12-04T09:31:29.0281470Z creating: dist/ 2025-12-04T09:31:30.4512751Z inflating: dist/torch-2.10.0a0+gitffd9b0f-cp310-cp310-linux_x86_64.whl 2025-12-04T09:31:30.4513349Z creating: dist/vision/ 2025-12-04T09:31:30.4611904Z inflating: dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl 2025-12-04T09:31:30.4612540Z creating: dist/audio/ 2025-12-04T09:31:30.4643926Z inflating: dist/audio/torchaudio-2.10.0a0+e90a398-cp310-cp310-linux_x86_64.whl 2025-12-04T09:31:30.4644535Z creating: dist/ao/ 2025-12-04T09:31:30.4694162Z inflating: dist/ao/torchao-0.7.0+git51c87b6e-py3-none-any.whl 2025-12-04T09:31:30.4831259Z inflating: dist/.ninja_log 2025-12-04T09:31:30.4831695Z creating: build/custom_test_artifacts/ 2025-12-04T09:31:30.4832169Z creating: build/custom_test_artifacts/custom-op-build/ 2025-12-04T09:31:30.4832760Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2025-12-04T09:31:30.4833774Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2025-12-04T09:31:30.4841422Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2025-12-04T09:31:30.4842250Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/ 2025-12-04T09:31:30.4843057Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CMakeSystem.cmake 2025-12-04T09:31:30.4844422Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdC/ 2025-12-04T09:31:30.4845280Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdC/tmp/ 2025-12-04T09:31:30.4846462Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdC/CMakeCCompilerId.c 2025-12-04T09:31:30.4847459Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdC/a.out 2025-12-04T09:31:30.4848377Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CMakeCCompiler.cmake 2025-12-04T09:31:30.4849279Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdCXX/ 2025-12-04T09:31:30.4850144Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdCXX/tmp/ 2025-12-04T09:31:30.4851632Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-12-04T09:31:30.4853005Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CompilerIdCXX/a.out 2025-12-04T09:31:30.4854030Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CMakeCXXCompiler.cmake 2025-12-04T09:31:30.4855609Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_C.bin 2025-12-04T09:31:30.4857527Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_CXX.bin 2025-12-04T09:31:30.4858452Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2025-12-04T09:31:30.4859216Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2025-12-04T09:31:30.4860002Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2025-12-04T09:31:30.4860873Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2025-12-04T09:31:30.4861881Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2025-12-04T09:31:30.4863111Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2025-12-04T09:31:30.4863999Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2025-12-04T09:31:30.4864924Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2025-12-04T09:31:30.4865842Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2025-12-04T09:31:30.4866859Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2025-12-04T09:31:30.4867977Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2025-12-04T09:31:30.4868912Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2025-12-04T09:31:30.4885929Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2025-12-04T09:31:30.5113922Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2025-12-04T09:31:30.5114831Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2025-12-04T09:31:30.5115795Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2025-12-04T09:31:30.5117082Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2025-12-04T09:31:30.5118113Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2025-12-04T09:31:30.5119175Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2025-12-04T09:31:30.5120143Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2025-12-04T09:31:30.5121256Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2025-12-04T09:31:30.5122233Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2025-12-04T09:31:30.5123211Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2025-12-04T09:31:30.5124176Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2025-12-04T09:31:30.5140948Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2025-12-04T09:31:30.5235894Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2025-12-04T09:31:30.5236987Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-12-04T09:31:30.5237953Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2025-12-04T09:31:30.5238938Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2025-12-04T09:31:30.5239705Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2025-12-04T09:31:30.5240461Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2025-12-04T09:31:30.5241472Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2025-12-04T09:31:30.5242209Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2025-12-04T09:31:30.5242867Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2025-12-04T09:31:30.5442329Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2025-12-04T09:31:30.5503914Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2025-12-04T09:31:30.5504568Z creating: build/custom_test_artifacts/jit-hook-build/ 2025-12-04T09:31:30.5505147Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2025-12-04T09:31:30.5505844Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2025-12-04T09:31:30.5508756Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2025-12-04T09:31:30.5509564Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/ 2025-12-04T09:31:30.5510533Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CMakeSystem.cmake 2025-12-04T09:31:30.5511404Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdC/ 2025-12-04T09:31:30.5512240Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdC/tmp/ 2025-12-04T09:31:30.5513201Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdC/CMakeCCompilerId.c 2025-12-04T09:31:30.5514276Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdC/a.out 2025-12-04T09:31:30.5515189Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CMakeCCompiler.cmake 2025-12-04T09:31:30.5516437Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdCXX/ 2025-12-04T09:31:30.5517290Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdCXX/tmp/ 2025-12-04T09:31:30.5518666Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-12-04T09:31:30.5519820Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CompilerIdCXX/a.out 2025-12-04T09:31:30.5520747Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CMakeCXXCompiler.cmake 2025-12-04T09:31:30.5522373Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_C.bin 2025-12-04T09:31:30.5526860Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_CXX.bin 2025-12-04T09:31:30.5527990Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2025-12-04T09:31:30.5528803Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2025-12-04T09:31:30.5529702Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2025-12-04T09:31:30.5530796Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2025-12-04T09:31:30.5531860Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2025-12-04T09:31:30.5532886Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2025-12-04T09:31:30.5533825Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2025-12-04T09:31:30.5534788Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2025-12-04T09:31:30.5536170Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2025-12-04T09:31:30.5537122Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2025-12-04T09:31:30.5538069Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2025-12-04T09:31:30.5538988Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2025-12-04T09:31:30.5553441Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2025-12-04T09:31:30.5628171Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2025-12-04T09:31:30.5629254Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-12-04T09:31:30.5630198Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2025-12-04T09:31:30.5631069Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2025-12-04T09:31:30.5631854Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2025-12-04T09:31:30.5633242Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2025-12-04T09:31:30.5633994Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2025-12-04T09:31:30.5634653Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2025-12-04T09:31:30.5635337Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2025-12-04T09:31:30.5682077Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2025-12-04T09:31:30.5682703Z creating: build/custom_test_artifacts/custom-backend-build/ 2025-12-04T09:31:30.5683326Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2025-12-04T09:31:30.5684064Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2025-12-04T09:31:30.5686658Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2025-12-04T09:31:30.5687493Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/ 2025-12-04T09:31:30.5688326Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CMakeSystem.cmake 2025-12-04T09:31:30.5689225Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdC/ 2025-12-04T09:31:30.5690649Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdC/tmp/ 2025-12-04T09:31:30.5691644Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdC/CMakeCCompilerId.c 2025-12-04T09:31:30.5692667Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdC/a.out 2025-12-04T09:31:30.5693620Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CMakeCCompiler.cmake 2025-12-04T09:31:30.5694939Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdCXX/ 2025-12-04T09:31:30.5695843Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdCXX/tmp/ 2025-12-04T09:31:30.5696895Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdCXX/CMakeCXXCompilerId.cpp 2025-12-04T09:31:30.5698169Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CompilerIdCXX/a.out 2025-12-04T09:31:30.5699157Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CMakeCXXCompiler.cmake 2025-12-04T09:31:30.5700430Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_C.bin 2025-12-04T09:31:30.5703050Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.6/CMakeDetermineCompilerABI_CXX.bin 2025-12-04T09:31:30.5704072Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2025-12-04T09:31:30.5704930Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2025-12-04T09:31:30.5705818Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2025-12-04T09:31:30.5707202Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2025-12-04T09:31:30.5708357Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2025-12-04T09:31:30.5710070Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2025-12-04T09:31:30.5711177Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2025-12-04T09:31:30.5712222Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2025-12-04T09:31:30.5713309Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2025-12-04T09:31:30.5714377Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2025-12-04T09:31:30.5715811Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2025-12-04T09:31:30.5717107Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2025-12-04T09:31:30.5718481Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2025-12-04T09:31:30.5851276Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2025-12-04T09:31:30.5852345Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2025-12-04T09:31:30.5853382Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2025-12-04T09:31:30.5854569Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2025-12-04T09:31:30.5855684Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2025-12-04T09:31:30.5857165Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2025-12-04T09:31:30.5858677Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2025-12-04T09:31:30.5859779Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2025-12-04T09:31:30.5860861Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2025-12-04T09:31:30.5862251Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2025-12-04T09:31:30.5863448Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2025-12-04T09:31:30.5879030Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2025-12-04T09:31:30.5944445Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2025-12-04T09:31:30.5945621Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2025-12-04T09:31:30.5946621Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2025-12-04T09:31:30.5947768Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2025-12-04T09:31:30.5948609Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2025-12-04T09:31:30.5949451Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2025-12-04T09:31:30.5950266Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2025-12-04T09:31:30.5950995Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2025-12-04T09:31:30.5951727Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2025-12-04T09:31:30.6064184Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2025-12-04T09:31:30.6112902Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2025-12-04T09:31:30.6113514Z creating: build/lib/ 2025-12-04T09:31:30.6207912Z inflating: build/lib/libprotobuf-lite.a 2025-12-04T09:31:30.6714112Z inflating: build/lib/libprotobuf.a 2025-12-04T09:31:30.7277482Z inflating: build/lib/libprotoc.a 2025-12-04T09:31:30.7287386Z inflating: build/lib/libpthreadpool.a 2025-12-04T09:31:30.7296003Z inflating: build/lib/libcpuinfo.a 2025-12-04T09:31:30.7305468Z inflating: build/lib/libcpuinfo_internals.a 2025-12-04T09:31:30.7306015Z inflating: build/lib/libclog.a 2025-12-04T09:31:30.7328205Z inflating: build/lib/libpytorch_qnnpack.a 2025-12-04T09:31:30.7329558Z inflating: build/lib/libnnpack_reference_layers.a 2025-12-04T09:31:30.7545763Z inflating: build/lib/libmicrokernels-prod.a 2025-12-04T09:31:30.7565199Z inflating: build/lib/libnnpack.a 2025-12-04T09:31:30.8572222Z inflating: build/lib/libmicrokernels-all.a 2025-12-04T09:31:30.8657735Z inflating: build/lib/libgtest.a 2025-12-04T09:31:30.8676495Z inflating: build/lib/libgmock.a 2025-12-04T09:31:30.8677269Z inflating: build/lib/libgtest_main.a 2025-12-04T09:31:30.8678082Z inflating: build/lib/libgmock_main.a 2025-12-04T09:31:30.8778861Z inflating: build/lib/libXNNPACK.a 2025-12-04T09:31:30.8868095Z inflating: build/lib/libbenchmark.a 2025-12-04T09:31:30.8868591Z inflating: build/lib/libbenchmark_main.a 2025-12-04T09:31:30.8869047Z inflating: build/lib/libjitprofiling.a 2025-12-04T09:31:30.8877189Z inflating: build/lib/libittnotify.a 2025-12-04T09:31:30.8954844Z inflating: build/lib/libasmjit.a 2025-12-04T09:31:31.0280936Z inflating: build/lib/libfbgemm.a 2025-12-04T09:31:31.0314877Z inflating: build/lib/libtensorpipe_uv.a 2025-12-04T09:31:31.0938977Z inflating: build/lib/libtensorpipe.a 2025-12-04T09:31:31.1071230Z inflating: build/lib/libgloo.a 2025-12-04T09:31:31.1129817Z inflating: build/lib/libonnx_proto.a 2025-12-04T09:31:31.1935550Z inflating: build/lib/libonnx.a 2025-12-04T09:31:32.3484878Z inflating: build/lib/libdnnl.a 2025-12-04T09:31:32.3505326Z inflating: build/lib/libfmt.a 2025-12-04T09:31:32.3823314Z inflating: build/lib/libkineto.a 2025-12-04T09:31:32.3954873Z inflating: build/lib/libc10.so 2025-12-04T09:31:32.3955901Z inflating: build/lib/libtorch_global_deps.so 2025-12-04T09:31:35.9445678Z inflating: build/lib/libtorch_cpu.so 2025-12-04T09:31:35.9446478Z inflating: build/lib/libtorch.so 2025-12-04T09:31:35.9530164Z inflating: build/lib/libtorchbind_test.so 2025-12-04T09:31:35.9551037Z inflating: build/lib/libjitbackend_test.so 2025-12-04T09:31:35.9582432Z inflating: build/lib/libbackend_with_compiler.so 2025-12-04T09:31:35.9607399Z inflating: build/lib/libaoti_custom_ops.so 2025-12-04T09:31:35.9611706Z inflating: build/lib/libshm.so 2025-12-04T09:31:36.2257076Z inflating: build/lib/libtorch_python.so 2025-12-04T09:31:36.2297045Z inflating: build/lib/libnnapi_backend.so 2025-12-04T09:31:36.2297498Z creating: build/bin/ 2025-12-04T09:31:36.2297824Z creating: build/bin/CMakeFiles/ 2025-12-04T09:31:36.2298197Z inflating: build/bin/cmake_install.cmake 2025-12-04T09:31:36.2298613Z inflating: build/bin/CTestTestfile.cmake 2025-12-04T09:31:36.2831309Z inflating: build/bin/protoc-3.13.0.0 2025-12-04T09:31:36.3353629Z inflating: build/bin/protoc 2025-12-04T09:31:36.3426181Z inflating: build/bin/c10_AllocatorConfig_test 2025-12-04T09:31:36.3492575Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2025-12-04T09:31:36.3555674Z inflating: build/bin/c10_DeviceGuard_test 2025-12-04T09:31:36.3621287Z inflating: build/bin/c10_Device_test 2025-12-04T09:31:36.3698264Z inflating: build/bin/c10_DispatchKeySet_test 2025-12-04T09:31:36.3770197Z inflating: build/bin/c10_Scalar_test 2025-12-04T09:31:36.3831805Z inflating: build/bin/c10_StreamGuard_test 2025-12-04T09:31:36.3904161Z inflating: build/bin/c10_SymInt_test 2025-12-04T09:31:36.3972207Z inflating: build/bin/c10_InlineDeviceGuard_test 2025-12-04T09:31:36.4041157Z inflating: build/bin/c10_InlineStreamGuard_test 2025-12-04T09:31:36.4118648Z inflating: build/bin/c10_SizesAndStrides_test 2025-12-04T09:31:36.4203828Z inflating: build/bin/c10_cow_test 2025-12-04T09:31:36.4268577Z inflating: build/bin/c10_Bitset_test 2025-12-04T09:31:36.4339463Z inflating: build/bin/c10_ArrayRef_test 2025-12-04T09:31:36.4399928Z inflating: build/bin/c10_ConstexprCrc_test 2025-12-04T09:31:36.4462213Z inflating: build/bin/c10_DeadlockDetection_test 2025-12-04T09:31:36.4532673Z inflating: build/bin/c10_Enumerate_test 2025-12-04T09:31:36.4605943Z inflating: build/bin/c10_LeftRight_test 2025-12-04T09:31:36.4676328Z inflating: build/bin/c10_NetworkFlow_test 2025-12-04T09:31:36.4739610Z inflating: build/bin/c10_Half_test 2025-12-04T09:31:36.4804937Z inflating: build/bin/c10_IntrusiveList_test 2025-12-04T09:31:36.4871392Z inflating: build/bin/c10_Synchronized_test 2025-12-04T09:31:36.4945412Z inflating: build/bin/c10_ThreadLocal_test 2025-12-04T09:31:36.5006699Z inflating: build/bin/c10_Semaphore_test 2025-12-04T09:31:36.5075037Z inflating: build/bin/c10_bfloat16_test 2025-12-04T09:31:36.5149142Z inflating: build/bin/c10_accumulate_test 2025-12-04T09:31:36.5214356Z inflating: build/bin/c10_TypeIndex_test 2025-12-04T09:31:36.5277704Z inflating: build/bin/c10_exception_test 2025-12-04T09:31:36.5346312Z inflating: build/bin/c10_bit_cast_test 2025-12-04T09:31:36.5420351Z inflating: build/bin/c10_complex_math_test 2025-12-04T09:31:36.5481986Z inflating: build/bin/c10_error_test 2025-12-04T09:31:36.5543756Z inflating: build/bin/c10_flags_test 2025-12-04T09:31:36.5617093Z inflating: build/bin/c10_complex_test 2025-12-04T09:31:36.5684766Z inflating: build/bin/c10_irange_test 2025-12-04T09:31:36.5747747Z inflating: build/bin/c10_generic_math_test 2025-12-04T09:31:36.5808998Z inflating: build/bin/c10_nofatal_test 2025-12-04T09:31:36.5879165Z inflating: build/bin/c10_lazy_test 2025-12-04T09:31:36.5954488Z inflating: build/bin/c10_logging_test 2025-12-04T09:31:36.6148623Z inflating: build/bin/c10_intrusive_ptr_test 2025-12-04T09:31:36.6238884Z inflating: build/bin/c10_optional_test 2025-12-04T09:31:36.6304463Z inflating: build/bin/c10_registry_test 2025-12-04T09:31:36.6385428Z inflating: build/bin/c10_ordered_preserving_dict_test 2025-12-04T09:31:36.6453775Z inflating: build/bin/c10_ssize_test 2025-12-04T09:31:36.6634414Z inflating: build/bin/c10_small_vector_test 2025-12-04T09:31:36.6708379Z inflating: build/bin/c10_string_util_test 2025-12-04T09:31:36.6773648Z inflating: build/bin/c10_string_view_test 2025-12-04T09:31:36.6836655Z inflating: build/bin/c10_tempfile_test 2025-12-04T09:31:36.6894051Z inflating: build/bin/c10_intrusive_ptr_benchmark 2025-12-04T09:31:36.6964831Z inflating: build/bin/c10_typeid_test 2025-12-04T09:31:36.7657716Z inflating: build/bin/vec_test_all_types_DEFAULT 2025-12-04T09:31:36.8373676Z inflating: build/bin/vec_test_all_types_AVX512 2025-12-04T09:31:36.9097872Z inflating: build/bin/vec_test_all_types_AVX2 2025-12-04T09:31:36.9223608Z inflating: build/bin/test_aoti_abi_check 2025-12-04T09:31:36.9284666Z inflating: build/bin/test_vec_half_DEFAULT 2025-12-04T09:31:36.9346898Z inflating: build/bin/test_vec_half_AVX512 2025-12-04T09:31:36.9412277Z inflating: build/bin/test_vec_half_AVX2 2025-12-04T09:31:36.9483748Z inflating: build/bin/BackoffTest 2025-12-04T09:31:36.9548400Z inflating: build/bin/FileStoreTest 2025-12-04T09:31:36.9616622Z inflating: build/bin/static_runtime_bench 2025-12-04T09:31:36.9923921Z inflating: build/bin/static_runtime_test 2025-12-04T09:31:37.0016194Z inflating: build/bin/Dict_test 2025-12-04T09:31:37.0080828Z inflating: build/bin/Dimname_test 2025-12-04T09:31:37.0166295Z inflating: build/bin/MaybeOwned_test 2025-12-04T09:31:37.0240704Z inflating: build/bin/NamedTensor_test 2025-12-04T09:31:37.0311161Z inflating: build/bin/apply_utils_test 2025-12-04T09:31:37.0384198Z inflating: build/bin/atest 2025-12-04T09:31:37.0470969Z inflating: build/bin/basic 2025-12-04T09:31:37.0540150Z inflating: build/bin/broadcast_test 2025-12-04T09:31:37.0602928Z inflating: build/bin/cpu_allocator_test 2025-12-04T09:31:37.0677817Z inflating: build/bin/cpu_generator_test 2025-12-04T09:31:37.0748333Z inflating: build/bin/cpu_profiling_allocator_test 2025-12-04T09:31:37.0859633Z inflating: build/bin/cpu_rng_test 2025-12-04T09:31:37.0927786Z inflating: build/bin/dlconvertor_test 2025-12-04T09:31:37.1002966Z inflating: build/bin/extension_backend_test 2025-12-04T09:31:37.1069779Z inflating: build/bin/half_test 2025-12-04T09:31:37.1190980Z inflating: build/bin/ivalue_test 2025-12-04T09:31:37.1258369Z inflating: build/bin/lazy_tensor_test 2025-12-04T09:31:37.1324025Z inflating: build/bin/math_kernel_test 2025-12-04T09:31:37.1392741Z inflating: build/bin/memory_format_test 2025-12-04T09:31:37.1463922Z inflating: build/bin/memory_overlapping_test 2025-12-04T09:31:37.1530090Z inflating: build/bin/mobile_memory_cleanup 2025-12-04T09:31:37.1600173Z inflating: build/bin/native_test 2025-12-04T09:31:37.1690792Z inflating: build/bin/operator_name_test 2025-12-04T09:31:37.1735956Z inflating: build/bin/operators_test 2025-12-04T09:31:37.1798906Z inflating: build/bin/packedtensoraccessor_test 2025-12-04T09:31:37.1882520Z inflating: build/bin/pow_test 2025-12-04T09:31:37.1960532Z inflating: build/bin/quantized_test 2025-12-04T09:31:37.2022734Z inflating: build/bin/reduce_ops_test 2025-12-04T09:31:37.2085814Z inflating: build/bin/reportMemoryUsage_test 2025-12-04T09:31:37.2159603Z inflating: build/bin/scalar_tensor_test 2025-12-04T09:31:37.2234416Z inflating: build/bin/scalar_test 2025-12-04T09:31:37.2298584Z inflating: build/bin/StorageUtils_test 2025-12-04T09:31:37.2362585Z inflating: build/bin/stride_properties_test 2025-12-04T09:31:37.2466485Z inflating: build/bin/tensor_iterator_test 2025-12-04T09:31:37.2534623Z inflating: build/bin/test_parallel 2025-12-04T09:31:37.2595719Z inflating: build/bin/thread_init_test 2025-12-04T09:31:37.2668859Z inflating: build/bin/type_ptr_test 2025-12-04T09:31:37.2746839Z inflating: build/bin/type_test 2025-12-04T09:31:37.2812014Z inflating: build/bin/undefined_tensor_test 2025-12-04T09:31:37.2873272Z inflating: build/bin/verify_api_visibility 2025-12-04T09:31:37.2968694Z inflating: build/bin/legacy_vmap_test 2025-12-04T09:31:37.3032139Z inflating: build/bin/weakref_test 2025-12-04T09:31:37.3096142Z inflating: build/bin/wrapdim_test 2025-12-04T09:31:37.3164276Z inflating: build/bin/xla_tensor_test 2025-12-04T09:31:37.3242469Z inflating: build/bin/IListRef_test 2025-12-04T09:31:37.3367507Z inflating: build/bin/List_test 2025-12-04T09:31:37.3456137Z inflating: build/bin/KernelFunction_test 2025-12-04T09:31:37.3597900Z inflating: build/bin/kernel_function_legacy_test 2025-12-04T09:31:37.3723313Z inflating: build/bin/kernel_function_test 2025-12-04T09:31:37.3870595Z inflating: build/bin/kernel_lambda_legacy_test 2025-12-04T09:31:37.4002902Z inflating: build/bin/kernel_lambda_test 2025-12-04T09:31:37.4075213Z inflating: build/bin/kernel_stackbased_test 2025-12-04T09:31:37.4193457Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2025-12-04T09:31:37.4263209Z inflating: build/bin/CppSignature_test 2025-12-04T09:31:37.4330632Z inflating: build/bin/backend_fallback_test 2025-12-04T09:31:37.4395034Z inflating: build/bin/op_allowlist_test 2025-12-04T09:31:37.4766676Z inflating: build/bin/op_registration_test 2025-12-04T09:31:37.4850076Z inflating: build/bin/inline_container_test 2025-12-04T09:31:37.5267316Z inflating: build/bin/test_lazy 2025-12-04T09:31:37.5337774Z inflating: build/bin/TCPStoreTest 2025-12-04T09:31:37.5407911Z inflating: build/bin/HashStoreTest 2025-12-04T09:31:37.6715078Z inflating: build/bin/test_jit 2025-12-04T09:31:37.6796805Z inflating: build/bin/ProcessGroupGlooTest 2025-12-04T09:31:37.6868392Z inflating: build/bin/test_aoti_inference 2025-12-04T09:31:37.6870884Z inflating: build/bin/example_allreduce 2025-12-04T09:31:37.6945610Z inflating: build/bin/test_dist_autograd 2025-12-04T09:31:37.7033006Z inflating: build/bin/test_cpp_rpc 2025-12-04T09:31:37.7035394Z inflating: build/bin/parallel_benchmark 2025-12-04T09:31:37.8415352Z inflating: build/bin/test_api 2025-12-04T09:31:37.8418255Z inflating: build/bin/torch_shm_manager 2025-12-04T09:31:37.8418662Z creating: .additional_ci_files/ 2025-12-04T09:31:37.8497357Z inflating: .additional_ci_files/test-times.json 2025-12-04T09:31:37.8768418Z inflating: .additional_ci_files/test-class-times.json 2025-12-04T09:31:37.8837787Z ##[group]Run rm artifacts.zip 2025-12-04T09:31:37.8838168Z rm artifacts.zip 2025-12-04T09:31:37.8845084Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:37.8845535Z env: 2025-12-04T09:31:37.8845776Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:37.8846094Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:37.8846386Z ##[endgroup] 2025-12-04T09:31:37.9289626Z ##[group]Run df -H 2025-12-04T09:31:37.9289902Z df -H 2025-12-04T09:31:37.9295765Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:37.9296231Z env: 2025-12-04T09:31:37.9296490Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:37.9296795Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:37.9297091Z ##[endgroup] 2025-12-04T09:31:37.9346626Z Filesystem Size Used Avail Use% Mounted on 2025-12-04T09:31:37.9347419Z devtmpfs 4.2M 0 4.2M 0% /dev 2025-12-04T09:31:37.9347911Z tmpfs 85G 0 85G 0% /dev/shm 2025-12-04T09:31:37.9348356Z tmpfs 34G 648k 34G 1% /run 2025-12-04T09:31:37.9348984Z /dev/xvda1 215G 72G 144G 34% / 2025-12-04T09:31:37.9349349Z tmpfs 85G 13k 85G 1% /tmp 2025-12-04T09:31:37.9349749Z /dev/xvda128 11M 1.4M 9.2M 13% /boot/efi 2025-12-04T09:31:37.9388193Z Prepare all required actions 2025-12-04T09:31:37.9396692Z Getting action download info 2025-12-04T09:31:38.1594539Z ##[group]Run ./.github/actions/download-td-artifacts 2025-12-04T09:31:38.1594979Z with: 2025-12-04T09:31:38.1595221Z env: 2025-12-04T09:31:38.1595460Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:38.1595784Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:38.1596079Z ##[endgroup] 2025-12-04T09:31:38.1662451Z ##[group]Run seemethere/download-artifact-s3@v4 2025-12-04T09:31:38.1662858Z with: 2025-12-04T09:31:38.1663093Z name: td_results 2025-12-04T09:31:38.1663375Z s3-bucket: gha-artifacts 2025-12-04T09:31:38.1663686Z region: us-east-1 2025-12-04T09:31:38.1663938Z env: 2025-12-04T09:31:38.1664184Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:38.1664497Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:38.1664804Z ##[endgroup] 2025-12-04T09:31:38.9510687Z (node:57777) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2025-12-04T09:31:38.9511483Z 2025-12-04T09:31:38.9511739Z Please migrate your code to use AWS SDK for JavaScript (v3). 2025-12-04T09:31:38.9512402Z For more information, check the migration guide at https://a.co/7PzMCcy 2025-12-04T09:31:38.9513146Z (Use `node --trace-warnings ...` to show where the warning was created) 2025-12-04T09:31:39.0562152Z Found 0 objects with prefix pytorch/pytorch/19923066595/td_results/ 2025-12-04T09:31:39.0567816Z Artifact download has finished successfully 2025-12-04T09:31:39.1754970Z ##[group]Run mkdir -p .additional_ci_files 2025-12-04T09:31:39.1755415Z mkdir -p .additional_ci_files 2025-12-04T09:31:39.1755940Z mv td_results.json .additional_ci_files/td_results.json || true 2025-12-04T09:31:39.1767082Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:39.1767554Z env: 2025-12-04T09:31:39.1767820Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:39.1768153Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:39.1768443Z ##[endgroup] 2025-12-04T09:31:39.1827718Z mv: cannot stat 'td_results.json': No such file or directory 2025-12-04T09:31:39.1887925Z ##[group]Run .github/scripts/parse_ref.py 2025-12-04T09:31:39.1888403Z .github/scripts/parse_ref.py 2025-12-04T09:31:39.1894361Z shell: /usr/bin/bash -e {0} 2025-12-04T09:31:39.1894694Z env: 2025-12-04T09:31:39.1894958Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:39.1895284Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:39.1895568Z ##[endgroup] 2025-12-04T09:31:39.2455083Z Setting output branch=main 2025-12-04T09:31:39.2606974Z Prepare all required actions 2025-12-04T09:31:39.2607438Z Getting action download info 2025-12-04T09:31:39.4151710Z ##[group]Run ./.github/actions/filter-test-configs 2025-12-04T09:31:39.4152134Z with: 2025-12-04T09:31:39.4152575Z github-token: *** 2025-12-04T09:31:39.4166627Z test-matrix: {"include": [{"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_avx2_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}]} 2025-12-04T09:31:39.4180834Z job-name: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:31:39.4181571Z env: 2025-12-04T09:31:39.4181825Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:39.4182274Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:39.4182555Z ##[endgroup] 2025-12-04T09:31:39.4233504Z ##[group]Run nick-fields/retry@v3.0.0 2025-12-04T09:31:39.4233865Z with: 2025-12-04T09:31:39.4234117Z shell: bash 2025-12-04T09:31:39.4234391Z timeout_minutes: 10 2025-12-04T09:31:39.4234681Z max_attempts: 5 2025-12-04T09:31:39.4234966Z retry_wait_seconds: 30 2025-12-04T09:31:39.4236202Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.2 2025-12-04T09:31:39.4237235Z polling_interval_seconds: 1 2025-12-04T09:31:39.4237561Z warning_on_retry: true 2025-12-04T09:31:39.4237882Z continue_on_error: false 2025-12-04T09:31:39.4238195Z env: 2025-12-04T09:31:39.4238436Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:39.4238755Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:39.4239337Z GITHUB_TOKEN: *** 2025-12-04T09:31:39.4239617Z ##[endgroup] 2025-12-04T09:31:39.5541617Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.2 2025-12-04T09:31:39.8454529Z Defaulting to user installation because normal site-packages is not writeable 2025-12-04T09:31:40.0138487Z Collecting requests==2.27.1 2025-12-04T09:31:40.0325572Z Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB) 2025-12-04T09:31:40.2959886Z Collecting pyyaml==6.0.2 2025-12-04T09:31:40.3042979Z Downloading PyYAML-6.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (737 kB) 2025-12-04T09:31:40.8786511Z Collecting charset-normalizer~=2.0.0 2025-12-04T09:31:40.8830097Z Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2025-12-04T09:31:40.9117803Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2025-12-04T09:31:40.9121957Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2025-12-04T09:31:40.9913410Z Collecting certifi>=2017.4.17 2025-12-04T09:31:40.9961756Z Downloading certifi-2025.11.12-py3-none-any.whl (159 kB) 2025-12-04T09:31:41.1388406Z Installing collected packages: charset-normalizer, certifi, requests, pyyaml 2025-12-04T09:31:41.4788973Z Successfully installed certifi-2025.11.12 charset-normalizer-2.0.12 pyyaml-6.0.2 requests-2.27.1 2025-12-04T09:31:42.5170465Z Command completed after 1 attempt(s). 2025-12-04T09:31:42.5239180Z ##[group]Run set -x 2025-12-04T09:31:42.5239467Z set -x 2025-12-04T09:31:42.5239736Z  2025-12-04T09:31:42.5240190Z # Use relative path here as this could be checked out anywhere, not necessarily 2025-12-04T09:31:42.5240744Z # in runner workspace 2025-12-04T09:31:42.5241201Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2025-12-04T09:31:42.5248061Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:42.5248506Z env: 2025-12-04T09:31:42.5248748Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:42.5249064Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:42.5249375Z ##[endgroup] 2025-12-04T09:31:42.5277389Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2025-12-04T09:31:42.5499895Z Setting output branch=main 2025-12-04T09:31:42.5560616Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2025-12-04T09:31:42.5561134Z echo "Workflow: ${GITHUB_WORKFLOW}" 2025-12-04T09:31:42.5561538Z echo "Job name: ${JOB_NAME}" 2025-12-04T09:31:42.5561890Z  2025-12-04T09:31:42.5562322Z # Use relative path here as this could be checked out anywhere, not necessarily 2025-12-04T09:31:42.5562891Z # in runner workspace 2025-12-04T09:31:42.5563399Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2025-12-04T09:31:42.5563966Z  --workflow "${GITHUB_WORKFLOW}" \ 2025-12-04T09:31:42.5564342Z  --job-name "${JOB_NAME}" \ 2025-12-04T09:31:42.5583679Z  --test-matrix "{"include": [{"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_avx2_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}]}" \ 2025-12-04T09:31:42.5598247Z  --selected-test-configs "" \ 2025-12-04T09:31:42.5598658Z  --pr-number "${PR_NUMBER}" \ 2025-12-04T09:31:42.5599137Z  --tag "${TAG}" \ 2025-12-04T09:31:42.5599485Z  --event-name "${EVENT_NAME}" \ 2025-12-04T09:31:42.5599864Z  --schedule "${SCHEDULE}" \ 2025-12-04T09:31:42.5600213Z  --branch "${HEAD_BRANCH}" 2025-12-04T09:31:42.5607223Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:42.5607684Z env: 2025-12-04T09:31:42.5608079Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:42.5608411Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:42.5609349Z GITHUB_TOKEN: *** 2025-12-04T09:31:42.5610075Z JOB_NAME: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:31:42.5610834Z PR_NUMBER: 2025-12-04T09:31:42.5611112Z TAG: 2025-12-04T09:31:42.5611362Z EVENT_NAME: schedule 2025-12-04T09:31:42.5611655Z SCHEDULE: 45 0,4,8,12,16,20 * * 1-5 2025-12-04T09:31:42.5612006Z HEAD_BRANCH: main 2025-12-04T09:31:42.5612282Z ##[endgroup] 2025-12-04T09:31:42.5638925Z Workflow: inductor-periodic 2025-12-04T09:31:42.5639734Z Job name: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:31:42.7885464Z Setting output keep-going=True 2025-12-04T09:31:42.7885906Z Setting output ci-verbose-test-logs=False 2025-12-04T09:31:42.7886336Z Setting output ci-test-showlocals=False 2025-12-04T09:31:42.7886772Z Setting output ci-no-test-timeout=False 2025-12-04T09:31:42.7887149Z Setting output ci-no-td=False 2025-12-04T09:31:42.7887518Z Setting output ci-td-distributed=False 2025-12-04T09:31:42.7887910Z Setting output is-unstable=False 2025-12-04T09:31:42.7888278Z Setting output reenabled-issues= 2025-12-04T09:31:42.7903286Z Setting output test-matrix={"include": [{"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_avx2_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}]} 2025-12-04T09:31:42.7917818Z Setting output is-test-matrix-empty=False 2025-12-04T09:31:42.8081327Z ##[group]Run echo "Filtered matrix:" 2025-12-04T09:31:42.8081773Z echo "Filtered matrix:" 2025-12-04T09:31:42.8095344Z echo "{"include": [{"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_avx2_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 1, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_avx2_timm", "shard": 2, "num_shards": 2, "runner": "linux.10xlarge.avx2"}, {"config": "cpu_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_timm", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.8xlarge.amx"}, {"config": "dynamic_cpu_aot_inductor_amp_freezing_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.8xlarge.amx"}]}" 2025-12-04T09:31:42.8109988Z  2025-12-04T09:31:42.8110244Z echo 2025-12-04T09:31:42.8110577Z echo "Is the current job unstable? False" 2025-12-04T09:31:42.8110964Z  2025-12-04T09:31:42.8111205Z echo 2025-12-04T09:31:42.8111514Z echo "Is keep-going label set? True" 2025-12-04T09:31:42.8111882Z  2025-12-04T09:31:42.8112123Z echo 2025-12-04T09:31:42.8112405Z echo "Reenabled issues? " 2025-12-04T09:31:42.8118933Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:42.8119375Z env: 2025-12-04T09:31:42.8119628Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:42.8119942Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:42.8120251Z ##[endgroup] 2025-12-04T09:31:42.8146780Z Filtered matrix: 2025-12-04T09:31:42.8163909Z {include: [{config: cpu_inductor_huggingface, shard: 1, num_shards: 1, runner: linux.8xlarge.amx}, {config: cpu_inductor_timm, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_timm, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_inductor_huggingface, shard: 1, num_shards: 1, runner: linux.8xlarge.amx}, {config: dynamic_cpu_inductor_timm, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_inductor_timm, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_freezing_avx2_huggingface, shard: 1, num_shards: 1, runner: linux.10xlarge.avx2}, {config: cpu_inductor_freezing_avx2_torchbench, shard: 1, num_shards: 2, runner: linux.10xlarge.avx2}, {config: cpu_inductor_freezing_avx2_torchbench, shard: 2, num_shards: 2, runner: linux.10xlarge.avx2}, {config: cpu_inductor_freezing_avx2_timm, shard: 1, num_shards: 2, runner: linux.10xlarge.avx2}, {config: cpu_inductor_freezing_avx2_timm, shard: 2, num_shards: 2, runner: linux.10xlarge.avx2}, {config: cpu_inductor_freezing_huggingface, shard: 1, num_shards: 1, runner: linux.8xlarge.amx}, {config: cpu_inductor_freezing_timm, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_freezing_timm, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_amp_freezing_huggingface, shard: 1, num_shards: 1, runner: linux.8xlarge.amx}, {config: cpu_inductor_amp_freezing_timm, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_amp_freezing_timm, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_amp_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_inductor_amp_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_freezing_huggingface, shard: 1, num_shards: 1, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_freezing_timm, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_freezing_timm, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_amp_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: cpu_aot_inductor_amp_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_aot_inductor_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_aot_inductor_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_aot_inductor_amp_freezing_torchbench, shard: 1, num_shards: 2, runner: linux.8xlarge.amx}, {config: dynamic_cpu_aot_inductor_amp_freezing_torchbench, shard: 2, num_shards: 2, runner: linux.8xlarge.amx}]} 2025-12-04T09:31:42.8177332Z 2025-12-04T09:31:42.8177467Z Is the current job unstable? False 2025-12-04T09:31:42.8177709Z 2025-12-04T09:31:42.8177850Z Is keep-going label set? True 2025-12-04T09:31:42.8178069Z 2025-12-04T09:31:42.8178197Z Reenabled issues? 2025-12-04T09:31:42.8269587Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2025-12-04T09:31:42.8270223Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2025-12-04T09:31:42.8276187Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:42.8276627Z env: 2025-12-04T09:31:42.8276891Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:42.8277213Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:42.8277498Z JOB_TIMEOUT: 240 2025-12-04T09:31:42.8277772Z ##[endgroup] 2025-12-04T09:31:42.8362069Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:31:42.8362710Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:31:42.8363246Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2025-12-04T09:31:42.8368987Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:31:42.8369433Z env: 2025-12-04T09:31:42.8369684Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:42.8369998Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:42.8370275Z ##[endgroup] 2025-12-04T09:31:42.8510831Z ##[group]Run set -x 2025-12-04T09:31:42.8511210Z set -x 2025-12-04T09:31:42.8511450Z  2025-12-04T09:31:42.8511741Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2025-12-04T09:31:42.8512209Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2025-12-04T09:31:42.8512663Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2025-12-04T09:31:42.8513093Z  TEST_COMMAND=.ci/onnx/test.sh 2025-12-04T09:31:42.8513445Z else 2025-12-04T09:31:42.8513731Z  TEST_COMMAND=.ci/pytorch/test.sh 2025-12-04T09:31:42.8514077Z fi 2025-12-04T09:31:42.8514310Z  2025-12-04T09:31:42.8514614Z # Leaving 1GB for the runner and other things 2025-12-04T09:31:42.8515302Z TOTAL_AVAILABLE_MEMORY_IN_GB=$(awk '/MemTotal/ { printf "%.3f \n", $2/1024/1024 - 1 }' /proc/meminfo) 2025-12-04T09:31:42.8516344Z # https://docs.docker.com/engine/containers/resource_constraints/#--memory-swap-details, the 3GB swap 2025-12-04T09:31:42.8517178Z # comes from https://github.com/pytorch/test-infra/pull/6058 2025-12-04T09:31:42.8517798Z TOTAL_MEMORY_WITH_SWAP=$(("${TOTAL_AVAILABLE_MEMORY_IN_GB%.*}" + 3)) 2025-12-04T09:31:42.8518293Z  2025-12-04T09:31:42.8518700Z if [[ ${BUILD_ENVIRONMENT} == *"s390x"* ]]; then 2025-12-04T09:31:42.8519089Z  SHM_OPTS= 2025-12-04T09:31:42.8519361Z  JENKINS_USER= 2025-12-04T09:31:42.8519753Z  # ensure that docker container cleanly exits in 12 hours 2025-12-04T09:31:42.8520295Z  # if for some reason cleanup action doesn't stop container 2025-12-04T09:31:42.8520759Z  # when job is cancelled 2025-12-04T09:31:42.8521116Z  DOCKER_SHELL_CMD="sleep 12h" 2025-12-04T09:31:42.8521830Z  USED_IMAGE="${DOCKER_IMAGE_S390X}" 2025-12-04T09:31:42.8522176Z else 2025-12-04T09:31:42.8522452Z  SHM_OPTS="--shm-size=${SHM_SIZE}" 2025-12-04T09:31:42.8522834Z  JENKINS_USER="--user jenkins" 2025-12-04T09:31:42.8523181Z  DOCKER_SHELL_CMD= 2025-12-04T09:31:42.8523503Z  USED_IMAGE="${DOCKER_IMAGE}" 2025-12-04T09:31:42.8523971Z fi 2025-12-04T09:31:42.8524186Z  2025-12-04T09:31:42.8524569Z # detached container should get cleaned up by teardown_ec2_linux 2025-12-04T09:31:42.8525183Z # TODO: Stop building test binaries as part of the build phase 2025-12-04T09:31:42.8525894Z # Used for GPU_FLAG, SHM_OPTS, JENKINS_USER and DOCKER_SHELL_CMD since that doesn't play nice 2025-12-04T09:31:42.8526501Z # shellcheck disable=SC2086,SC2090 2025-12-04T09:31:42.8526877Z container_name=$(docker run \ 2025-12-04T09:31:42.8527227Z  ${GPU_FLAG:-} \ 2025-12-04T09:31:42.8527562Z  ${SCCACHE_SERVER_PORT_DOCKER_FLAG:-} \ 2025-12-04T09:31:42.8527954Z  -e BUILD_ENVIRONMENT \ 2025-12-04T09:31:42.8528284Z  -e PR_NUMBER \ 2025-12-04T09:31:42.8528592Z  -e GITHUB_ACTIONS \ 2025-12-04T09:31:42.8528909Z  -e GITHUB_REPOSITORY \ 2025-12-04T09:31:42.8529241Z  -e GITHUB_WORKFLOW \ 2025-12-04T09:31:42.8529554Z  -e GITHUB_JOB \ 2025-12-04T09:31:42.8529840Z  -e GITHUB_RUN_ID \ 2025-12-04T09:31:42.8530153Z  -e GITHUB_RUN_NUMBER \ 2025-12-04T09:31:42.8530483Z  -e GITHUB_RUN_ATTEMPT \ 2025-12-04T09:31:42.8530797Z  -e JOB_ID \ 2025-12-04T09:31:42.8531069Z  -e JOB_NAME \ 2025-12-04T09:31:42.8531352Z  -e BASE_SHA \ 2025-12-04T09:31:42.8531632Z  -e BRANCH \ 2025-12-04T09:31:42.8531895Z  -e SHA1 \ 2025-12-04T09:31:42.8532315Z  -e AWS_DEFAULT_REGION \ 2025-12-04T09:31:42.8532641Z  -e IN_WHEEL_TEST \ 2025-12-04T09:31:42.8532938Z  -e SHARD_NUMBER \ 2025-12-04T09:31:42.8533240Z  -e TEST_CONFIG \ 2025-12-04T09:31:42.8533546Z  -e NUM_TEST_SHARDS \ 2025-12-04T09:31:42.8533944Z  -e REENABLED_ISSUES \ 2025-12-04T09:31:42.8534278Z  -e CONTINUE_THROUGH_ERROR \ 2025-12-04T09:31:42.8534619Z  -e VERBOSE_TEST_LOGS \ 2025-12-04T09:31:42.8534933Z  -e TEST_SHOWLOCALS \ 2025-12-04T09:31:42.8535257Z  -e NO_TEST_TIMEOUT \ 2025-12-04T09:31:42.8535564Z  -e NO_TD \ 2025-12-04T09:31:42.8535837Z  -e TD_DISTRIBUTED \ 2025-12-04T09:31:42.8536140Z  -e PR_LABELS \ 2025-12-04T09:31:42.8536464Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2025-12-04T09:31:42.8536837Z  -e SCCACHE_BUCKET \ 2025-12-04T09:31:42.8537140Z  -e SCCACHE_REGION \ 2025-12-04T09:31:42.8537448Z  -e XLA_CUDA \ 2025-12-04T09:31:42.8537765Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2025-12-04T09:31:42.8538162Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2025-12-04T09:31:42.8538577Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2025-12-04T09:31:42.8538998Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2025-12-04T09:31:42.8539392Z  -e HUGGING_FACE_HUB_TOKEN \ 2025-12-04T09:31:42.8539910Z  -e VLLM_TEST_HUGGING_FACE_TOKEN \ 2025-12-04T09:31:42.8540295Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2025-12-04T09:31:42.8540662Z  -e DASHBOARD_TAG \ 2025-12-04T09:31:42.8540976Z  -e ARTIFACTS_FILE_SUFFIX \ 2025-12-04T09:31:42.8541588Z  --memory="${TOTAL_AVAILABLE_MEMORY_IN_GB%.*}g" \ 2025-12-04T09:31:42.8542128Z  --memory-swap="${TOTAL_MEMORY_WITH_SWAP}g" \ 2025-12-04T09:31:42.8542600Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2025-12-04T09:31:42.8543039Z  --security-opt seccomp=unconfined \ 2025-12-04T09:31:42.8543433Z  --cap-add=SYS_PTRACE \ 2025-12-04T09:31:42.8543760Z  --ipc=host \ 2025-12-04T09:31:42.8544031Z  ${SHM_OPTS} \ 2025-12-04T09:31:42.8544310Z  --tty \ 2025-12-04T09:31:42.8544572Z  --detach \ 2025-12-04T09:31:42.8544858Z  --name="${container_name}" \ 2025-12-04T09:31:42.8545201Z  ${JENKINS_USER} \ 2025-12-04T09:31:42.8545587Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2025-12-04T09:31:42.8546036Z  -w /var/lib/jenkins/workspace \ 2025-12-04T09:31:42.8546383Z  "${USED_IMAGE}" \ 2025-12-04T09:31:42.8546789Z  ${DOCKER_SHELL_CMD} 2025-12-04T09:31:42.8547089Z ) 2025-12-04T09:31:42.8547623Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2025-12-04T09:31:42.8548168Z  2025-12-04T09:31:42.8548466Z if [[ ${BUILD_ENVIRONMENT} == *"s390x"* ]]; then 2025-12-04T09:31:42.8549144Z  docker exec -t "${container_name}" sh -c "python3 -m pip install -r .ci/docker/requirements-ci.txt" 2025-12-04T09:31:42.8549779Z fi 2025-12-04T09:31:42.8550007Z  2025-12-04T09:31:42.8550575Z docker exec -t "${container_name}" sh -c "python3 -m pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2025-12-04T09:31:42.8556604Z shell: /usr/bin/bash -e {0} 2025-12-04T09:31:42.8556901Z env: 2025-12-04T09:31:42.8557135Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:31:42.8557740Z HAS_NVIDIA_GPU: false 2025-12-04T09:31:42.8558089Z BUILD_ENVIRONMENT: linux-jammy-py3.10-gcc11-build 2025-12-04T09:31:42.8558489Z PR_NUMBER: 2025-12-04T09:31:42.8558747Z GITHUB_REPOSITORY: pytorch/pytorch 2025-12-04T09:31:42.8559106Z GITHUB_WORKFLOW: inductor-periodic 2025-12-04T09:31:42.8559437Z GITHUB_JOB: test 2025-12-04T09:31:42.8559802Z GITHUB_RUN_ID: 19923066595 2025-12-04T09:31:42.8560251Z GITHUB_RUN_NUMBER: 67027 2025-12-04T09:31:42.8560542Z GITHUB_RUN_ATTEMPT: 1 2025-12-04T09:31:42.8560899Z JOB_ID: 57118563305 2025-12-04T09:31:42.8561553Z JOB_NAME: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:31:42.8562283Z BRANCH: main 2025-12-04T09:31:42.8562657Z SHA1: ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:31:42.8563083Z BASE_SHA: ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:31:42.8563526Z TEST_CONFIG: cpu_inductor_freezing_avx2_huggingface 2025-12-04T09:31:42.8563915Z SHARD_NUMBER: 1 2025-12-04T09:31:42.8564154Z NUM_TEST_SHARDS: 1 2025-12-04T09:31:42.8564417Z EXTRA_FLAGS: 2025-12-04T09:31:42.8564664Z OP_BENCHMARK_TESTS: 2025-12-04T09:31:42.8565021Z REENABLED_ISSUES: 2025-12-04T09:31:42.8565292Z CONTINUE_THROUGH_ERROR: True 2025-12-04T09:31:42.8565755Z VERBOSE_TEST_LOGS: False 2025-12-04T09:31:42.8566224Z TEST_SHOWLOCALS: False 2025-12-04T09:31:42.8566509Z NO_TEST_TIMEOUT: False 2025-12-04T09:31:42.8566790Z NO_TD: False 2025-12-04T09:31:42.8567055Z TD_DISTRIBUTED: False 2025-12-04T09:31:42.8567399Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2025-12-04T09:31:42.8567811Z SCCACHE_REGION: us-east-1 2025-12-04T09:31:42.8568102Z SHM_SIZE: 1g 2025-12-04T09:31:42.8569065Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:31:42.8570936Z DOCKER_IMAGE_S390X: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:31:42.8572016Z XLA_CUDA: 2025-12-04T09:31:42.8572417Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2025-12-04T09:31:42.8572935Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2025-12-04T09:31:42.8573296Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2025-12-04T09:31:42.8573637Z DASHBOARD_TAG: 2025-12-04T09:31:42.8574122Z VLLM_TEST_HUGGING_FACE_TOKEN: *** 2025-12-04T09:31:42.8574588Z HUGGING_FACE_HUB_TOKEN: *** 2025-12-04T09:31:42.8575052Z SCRIBE_GRAPHQL_ACCESS_TOKEN: *** 2025-12-04T09:31:42.8575861Z ARTIFACTS_FILE_SUFFIX: test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305 2025-12-04T09:31:42.8576514Z ##[endgroup] 2025-12-04T09:31:42.8602451Z + [[ cpu_inductor_freezing_avx2_huggingface == \m\u\l\t\i\g\p\u ]] 2025-12-04T09:31:42.8603009Z + [[ linux-jammy-py3.10-gcc11-build == *onnx* ]] 2025-12-04T09:31:42.8603422Z + TEST_COMMAND=.ci/pytorch/test.sh 2025-12-04T09:31:42.8606022Z ++ awk '/MemTotal/ { printf "%.3f \n", $2/1024/1024 - 1 }' /proc/meminfo 2025-12-04T09:31:42.8628103Z + TOTAL_AVAILABLE_MEMORY_IN_GB='156.355 ' 2025-12-04T09:31:42.8632747Z + TOTAL_MEMORY_WITH_SWAP=159 2025-12-04T09:31:42.8633133Z + [[ linux-jammy-py3.10-gcc11-build == *\s\3\9\0\x* ]] 2025-12-04T09:31:42.8633745Z + SHM_OPTS=--shm-size=1g 2025-12-04T09:31:42.8634076Z + JENKINS_USER='--user jenkins' 2025-12-04T09:31:42.8634400Z + DOCKER_SHELL_CMD= 2025-12-04T09:31:42.8635409Z + USED_IMAGE=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:31:42.8640081Z +++ nproc --ignore=2 2025-12-04T09:31:42.8707181Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=38 -e SCCACHE_BUCKET -e SCCACHE_REGION -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e VLLM_TEST_HUGGING_FACE_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG -e ARTIFACTS_FILE_SUFFIX --memory=156g --memory-swap=159g --env-file=/tmp/github_env_19923066595 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:33:01.6981067Z + container_name=77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:33:01.6981976Z + echo DOCKER_CONTAINER_ID=77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:33:01.6982658Z + [[ linux-jammy-py3.10-gcc11-build == *\s\3\9\0\x* ]] 2025-12-04T09:33:01.6986502Z ++ echo dist/torch-2.10.0a0+gitffd9b0f-cp310-cp310-linux_x86_64.whl 2025-12-04T09:33:01.6989258Z + docker exec -t 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe sh -c 'python3 -m pip install dist/torch-2.10.0a0+gitffd9b0f-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2025-12-04T09:33:02.2717537Z Processing ./dist/torch-2.10.0a0+gitffd9b0f-cp310-cp310-linux_x86_64.whl (from torch==2.10.0a0+gitffd9b0f) 2025-12-04T09:33:02.9137020Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (3.18.0) 2025-12-04T09:33:02.9141096Z Requirement already satisfied: typing-extensions>=4.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (4.15.0) 2025-12-04T09:33:02.9146026Z Requirement already satisfied: sympy>=1.13.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (1.13.3) 2025-12-04T09:33:02.9151709Z Requirement already satisfied: networkx>=2.5.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (2.8.8) 2025-12-04T09:33:02.9155750Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (3.1.6) 2025-12-04T09:33:02.9161151Z Requirement already satisfied: fsspec>=0.8.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (2025.10.0) 2025-12-04T09:33:02.9176645Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (3.3.0) 2025-12-04T09:33:02.9621778Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (1.22.4) 2025-12-04T09:33:02.9651897Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.3->torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (1.3.0) 2025-12-04T09:33:02.9714902Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.10.0a0+gitffd9b0f->torch==2.10.0a0+gitffd9b0f) (3.0.3) 2025-12-04T09:33:04.2259245Z Installing collected packages: torch 2025-12-04T09:33:15.9816675Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2025-12-04T09:33:15.9817864Z dall-e 0.1 requires torchvision, which is not installed. 2025-12-04T09:33:15.9818376Z effdet 0.4.1 requires torchvision, which is not installed. 2025-12-04T09:33:15.9818978Z python-doctr 1.0.0 requires torchvision>=0.15.0, which is not installed. 2025-12-04T09:33:15.9819705Z pytorch-labs-segment-anything-fast 0.2 requires torchao, which is not installed. 2025-12-04T09:33:15.9820606Z pytorch-labs-segment-anything-fast 0.2 requires torchvision>=0.17.0.dev20231026, which is not installed. 2025-12-04T09:33:15.9821839Z timm 1.0.22 requires torchvision, which is not installed. 2025-12-04T09:33:15.9822448Z Successfully installed torch-2.10.0a0+gitffd9b0f 2025-12-04T09:33:16.0895564Z + export TERM=vt100 2025-12-04T09:33:16.0895952Z + TERM=vt100 2025-12-04T09:33:16.0896559Z ++ dirname .ci/pytorch/test.sh 2025-12-04T09:33:16.0903267Z + source .ci/pytorch/common.sh 2025-12-04T09:33:16.0907146Z +++ dirname .ci/pytorch/common.sh 2025-12-04T09:33:16.0914754Z ++ source .ci/pytorch/common_utils.sh 2025-12-04T09:33:16.0921004Z +++ declare -f -t trap_add 2025-12-04T09:33:16.0922544Z ++ set -ex -o pipefail 2025-12-04T09:33:16.0922905Z ++ [[ linux-jammy-py3.10-gcc11-build == *rocm* ]] 2025-12-04T09:33:16.0923312Z ++ BUILD_TEST_LIBTORCH=0 2025-12-04T09:33:16.0926544Z ++ dirname .ci/pytorch/test.sh 2025-12-04T09:33:16.0933952Z + source .ci/pytorch/common-build.sh 2025-12-04T09:33:16.0935691Z ++ [[ linux-jammy-py3.10-gcc11-build != *win-* ]] 2025-12-04T09:33:16.0941816Z ++++ dirname .ci/pytorch/common-build.sh 2025-12-04T09:33:16.0949772Z +++ cd .ci/pytorch 2025-12-04T09:33:16.0950132Z +++ pwd -P 2025-12-04T09:33:16.0952180Z ++ script_dir=/var/lib/jenkins/workspace/.ci/pytorch 2025-12-04T09:33:16.0952689Z ++ [[ linux-jammy-py3.10-gcc11-build == *-pch* ]] 2025-12-04T09:33:16.0953105Z ++ which sccache 2025-12-04T09:33:16.0972203Z ++ [[ -z ossci-compiler-cache-circleci-v2 ]] 2025-12-04T09:33:16.0972652Z ++ sccache --stop-server 2025-12-04T09:33:16.1005638Z ++ true 2025-12-04T09:33:16.1005952Z ++ rm -f /var/lib/jenkins/sccache_error.log 2025-12-04T09:33:16.1014913Z ++ trap_add sccache_epilogue EXIT 2025-12-04T09:33:16.1015308Z ++ trap_add_cmd=sccache_epilogue 2025-12-04T09:33:16.1015642Z ++ shift 2025-12-04T09:33:16.1015895Z ++ for trap_add_name in "$@" 2025-12-04T09:33:16.1021994Z ++++ trap -p EXIT 2025-12-04T09:33:16.1024407Z +++ eval 'extract_trap_cmd ' 2025-12-04T09:33:16.1024735Z ++++ extract_trap_cmd 2025-12-04T09:33:16.1025013Z ++++ printf '%s\n' '' 2025-12-04T09:33:16.1025316Z +++ printf '%s\n' sccache_epilogue 2025-12-04T09:33:16.1026553Z ++ trap -- ' 2025-12-04T09:33:16.1027032Z sccache_epilogue' EXIT 2025-12-04T09:33:16.1027609Z ++ [[ -n 1 ]] 2025-12-04T09:33:16.1028362Z ++ echo 'Skipping sccache server initialization, setting environment variables' 2025-12-04T09:33:16.1029537Z Skipping sccache server initialization, setting environment variables 2025-12-04T09:33:16.1030327Z ++ export SCCACHE_IDLE_TIMEOUT=0 2025-12-04T09:33:16.1030864Z ++ SCCACHE_IDLE_TIMEOUT=0 2025-12-04T09:33:16.1031455Z ++ export SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 2025-12-04T09:33:16.1032210Z ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 2025-12-04T09:33:16.1040628Z ++ export RUST_LOG=sccache::server=error 2025-12-04T09:33:16.1041142Z ++ RUST_LOG=sccache::server=error 2025-12-04T09:33:16.1041644Z ++ sccache --zero-stats 2025-12-04T09:33:16.2289051Z Statistics zeroed. 2025-12-04T09:33:16.2301404Z ++ which ccache 2025-12-04T09:33:16.2328218Z + [[ linux-jammy-py3.10-gcc11-build != *rocm* ]] 2025-12-04T09:33:16.2328742Z + [[ linux-jammy-py3.10-gcc11-build != *s390x* ]] 2025-12-04T09:33:16.2329190Z + [[ -d /var/lib/jenkins/workspace ]] 2025-12-04T09:33:16.2329812Z ++ stat -c %u /var/lib/jenkins/workspace 2025-12-04T09:33:16.2347126Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2025-12-04T09:33:16.2347831Z + trap_add cleanup_workspace EXIT 2025-12-04T09:33:16.2348216Z + trap_add_cmd=cleanup_workspace 2025-12-04T09:33:16.2348541Z + shift 2025-12-04T09:33:16.2348807Z + for trap_add_name in "$@" 2025-12-04T09:33:16.2352412Z +++ trap -p EXIT 2025-12-04T09:33:16.2360295Z ++ eval 'extract_trap_cmd trap -- '\'' 2025-12-04T09:33:16.2360733Z sccache_epilogue'\'' EXIT' 2025-12-04T09:33:16.2361083Z +++ extract_trap_cmd trap -- ' 2025-12-04T09:33:16.2361412Z sccache_epilogue' EXIT 2025-12-04T09:33:16.2361712Z +++ printf '%s\n' ' 2025-12-04T09:33:16.2361997Z sccache_epilogue' 2025-12-04T09:33:16.2362284Z ++ printf '%s\n' cleanup_workspace 2025-12-04T09:33:16.2362633Z + trap -- ' 2025-12-04T09:33:16.2362890Z sccache_epilogue 2025-12-04T09:33:16.2363389Z cleanup_workspace' EXIT 2025-12-04T09:33:16.2363761Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2025-12-04T09:33:16.9988661Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2025-12-04T09:33:17.0008062Z + [[ linux-jammy-py3.10-gcc11-build == *cuda* ]] 2025-12-04T09:33:17.0008846Z + echo 'Environment variables:' 2025-12-04T09:33:17.0009208Z Environment variables: 2025-12-04T09:33:17.0009493Z + env 2025-12-04T09:33:17.0018417Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-12-04T09:33:17.0019369Z CONTINUE_THROUGH_ERROR=True 2025-12-04T09:33:17.0019785Z BUILD_ENVIRONMENT=linux-jammy-py3.10-gcc11-build 2025-12-04T09:33:17.0020713Z VLLM_TEST_HUGGING_FACE_TOKEN=*** 2025-12-04T09:33:17.0021229Z HOSTNAME=77ce1fdc01d3 2025-12-04T09:33:17.0021913Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_4909f104-ffeb-4dc5-9e8c-4243af229733 2025-12-04T09:33:17.0022649Z GITHUB_ACTION=__run_3 2025-12-04T09:33:17.0022976Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2025-12-04T09:33:17.0023328Z GITHUB_RUN_NUMBER=67027 2025-12-04T09:33:17.0023921Z TEST_CONFIG=cpu_inductor_freezing_avx2_huggingface 2025-12-04T09:33:17.0024558Z GITHUB_REPOSITORY_OWNER_ID=21003710 2025-12-04T09:33:17.0024945Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2025-12-04T09:33:17.0025303Z SCCACHE_IDLE_TIMEOUT=0 2025-12-04T09:33:17.0025811Z SCRIBE_GRAPHQL_ACCESS_TOKEN=*** 2025-12-04T09:33:17.0026186Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot 2025-12-04T09:33:17.0026822Z GITHUB_REF_TYPE=branch 2025-12-04T09:33:17.0027190Z BASE_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:33:17.0027603Z XLA_CUDA= 2025-12-04T09:33:17.0027865Z NCCL_LIB_DIR=/usr/local/cuda/lib64/ 2025-12-04T09:33:17.0028354Z HUGGING_FACE_HUB_TOKEN=*** 2025-12-04T09:33:17.0028894Z *** 2025-12-04T09:33:17.0029159Z GITHUB_REPOSITORY_ID=65600975 2025-12-04T09:33:17.0029483Z GITHUB_ACTIONS=true 2025-12-04T09:33:17.0029844Z SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 2025-12-04T09:33:17.0030331Z SHA1=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:33:17.0030776Z GITHUB_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:33:17.0031480Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/heads/main 2025-12-04T09:33:17.0032125Z UCC_HOME=/usr 2025-12-04T09:33:17.0032394Z VERBOSE_TEST_LOGS=False 2025-12-04T09:33:17.0032712Z GITHUB_REF=refs/heads/main 2025-12-04T09:33:17.0033033Z SHARD_NUMBER=1 2025-12-04T09:33:17.0033316Z GITHUB_REF_PROTECTED=true 2025-12-04T09:33:17.0033622Z HOME=/var/lib/jenkins 2025-12-04T09:33:17.0033958Z GITHUB_API_URL=https://api.github.com 2025-12-04T09:33:17.0034361Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2025-12-04T09:33:17.0034701Z UCX_COMMIT= 2025-12-04T09:33:17.0034962Z USE_SYSTEM_NCCL=1 2025-12-04T09:33:17.0035237Z NUM_TEST_SHARDS=1 2025-12-04T09:33:17.0035498Z UCX_HOME=/usr 2025-12-04T09:33:17.0036182Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_4909f104-ffeb-4dc5-9e8c-4243af229733 2025-12-04T09:33:17.0037382Z JOB_NAME=periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:33:17.0038523Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_4909f104-ffeb-4dc5-9e8c-4243af229733 2025-12-04T09:33:17.0039536Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2025-12-04T09:33:17.0040318Z GITHUB_EVENT_NAME=schedule 2025-12-04T09:33:17.0040635Z DASHBOARD_TAG= 2025-12-04T09:33:17.0040895Z GITHUB_RUN_ID=19923066595 2025-12-04T09:33:17.0041207Z INSTALLED_OPENBLAS= 2025-12-04T09:33:17.0041956Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_4909f104-ffeb-4dc5-9e8c-4243af229733 2025-12-04T09:33:17.0042786Z GITHUB_ACTOR=pytorchmergebot 2025-12-04T09:33:17.0043113Z PR_NUMBER= 2025-12-04T09:33:17.0043365Z DESIRED_CUDA= 2025-12-04T09:33:17.0043620Z GITHUB_RUN_ATTEMPT=1 2025-12-04T09:33:17.0043927Z ANACONDA_PYTHON_VERSION=3.10 2025-12-04T09:33:17.0044520Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2025-12-04T09:33:17.0044926Z TERM=vt100 2025-12-04T09:33:17.0045184Z INSTALLED_VISION=yes 2025-12-04T09:33:17.0045473Z BRANCH=main 2025-12-04T09:33:17.0045740Z SCCACHE_REGION=us-east-1 2025-12-04T09:33:17.0046138Z OPENSSL_ROOT_DIR=/opt/openssl 2025-12-04T09:33:17.0046485Z BUILD_AOT_INDUCTOR_TEST=yes 2025-12-04T09:33:17.0046817Z CUDA_PATH=/usr/local/cuda 2025-12-04T09:33:17.0047427Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2025-12-04T09:33:17.0048142Z GITHUB_SERVER_URL=https://github.com 2025-12-04T09:33:17.0048509Z UCC_COMMIT= 2025-12-04T09:33:17.0048753Z REENABLED_ISSUES= 2025-12-04T09:33:17.0049029Z DOCS=yes 2025-12-04T09:33:17.0049270Z SHLVL=1 2025-12-04T09:33:17.0049494Z MAX_JOBS=38 2025-12-04T09:33:17.0049752Z GITHUB_ACTOR_ID=97764156 2025-12-04T09:33:17.0050171Z GITHUB_WORKFLOW_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:33:17.0050629Z GITHUB_REF_NAME=main 2025-12-04T09:33:17.0051081Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2025-12-04T09:33:17.0051602Z GITHUB_JOB=test 2025-12-04T09:33:17.0051977Z NO_TEST_TIMEOUT=False 2025-12-04T09:33:17.0052267Z TD_DISTRIBUTED=False 2025-12-04T09:33:17.0052581Z GITHUB_REPOSITORY=pytorch/pytorch 2025-12-04T09:33:17.0052936Z GITHUB_RETENTION_DAYS=90 2025-12-04T09:33:17.0053229Z OPENSSL_DIR=/opt/openssl 2025-12-04T09:33:17.0053539Z GITHUB_ACTION_REPOSITORY= 2025-12-04T09:33:17.0054461Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-12-04T09:33:17.0055400Z GITHUB_BASE_REF= 2025-12-04T09:33:17.0055669Z INSTALLED_ACL= 2025-12-04T09:33:17.0056241Z ARTIFACTS_FILE_SUFFIX=test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305 2025-12-04T09:33:17.0056885Z CI=true 2025-12-04T09:33:17.0057140Z GITHUB_REPOSITORY_OWNER=pytorch 2025-12-04T09:33:17.0057519Z RUST_LOG=sccache::server=error 2025-12-04T09:33:17.0057823Z JOB_ID=57118563305 2025-12-04T09:33:17.0058091Z GITHUB_HEAD_REF= 2025-12-04T09:33:17.0058362Z GITHUB_ACTION_REF= 2025-12-04T09:33:17.0058699Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2025-12-04T09:33:17.0059098Z TEST_SHOWLOCALS=False 2025-12-04T09:33:17.0059421Z GITHUB_WORKFLOW=inductor-periodic 2025-12-04T09:33:17.0059783Z DEBIAN_FRONTEND=noninteractive 2025-12-04T09:33:17.0060505Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_4909f104-ffeb-4dc5-9e8c-4243af229733 2025-12-04T09:33:17.0061259Z NO_TD=False 2025-12-04T09:33:17.0061527Z SKIP_SCCACHE_INITIALIZATION=1 2025-12-04T09:33:17.0061870Z NCCL_INCLUDE_DIR=/usr/local/cuda/include/ 2025-12-04T09:33:17.0062245Z _=/usr/bin/env 2025-12-04T09:33:17.0062612Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2025-12-04T09:33:17.0389131Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2025-12-04T09:33:17.0390024Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2025-12-04T09:33:17.0390742Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2025-12-04T09:33:17.0391464Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2025-12-04T09:33:17.0392081Z + BUILD_DIR=build 2025-12-04T09:33:17.0392646Z + BUILD_RENAMED_DIR=build_renamed 2025-12-04T09:33:17.0393170Z + BUILD_BIN_DIR=build/bin 2025-12-04T09:33:17.0393484Z + SHARD_NUMBER=1 2025-12-04T09:33:17.0393748Z + NUM_TEST_SHARDS=1 2025-12-04T09:33:17.0394062Z + export TORCH_SERIALIZATION_DEBUG=1 2025-12-04T09:33:17.0394441Z + TORCH_SERIALIZATION_DEBUG=1 2025-12-04T09:33:17.0394765Z + export VALGRIND=ON 2025-12-04T09:33:17.0395058Z + VALGRIND=ON 2025-12-04T09:33:17.0395387Z + [[ linux-jammy-py3.10-gcc11-build == *clang9* ]] 2025-12-04T09:33:17.0395839Z + [[ linux-jammy-py3.10-gcc11-build == *xpu* ]] 2025-12-04T09:33:17.0396241Z + detect_cuda_arch 2025-12-04T09:33:17.0396574Z + [[ linux-jammy-py3.10-gcc11-build == *cuda* ]] 2025-12-04T09:33:17.0397269Z + [[ linux-jammy-py3.10-gcc11-build == *s390x* ]] 2025-12-04T09:33:17.0397671Z + [[ 0 == \1 ]] 2025-12-04T09:33:17.0397941Z + [[ True == \1 ]] 2025-12-04T09:33:17.0398255Z + [[ linux-jammy-py3.10-gcc11-build != *bazel* ]] 2025-12-04T09:33:17.0398809Z ++ realpath build/custom_test_artifacts 2025-12-04T09:33:17.0403176Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2025-12-04T09:33:17.0432298Z + [[ -n '' ]] 2025-12-04T09:33:17.0432780Z + echo 'Environment variables' 2025-12-04T09:33:17.0433399Z Environment variables 2025-12-04T09:33:17.0433900Z + env 2025-12-04T09:33:17.0434635Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-12-04T09:33:17.0435210Z CONTINUE_THROUGH_ERROR=True 2025-12-04T09:33:17.0435922Z BUILD_ENVIRONMENT=linux-jammy-py3.10-gcc11-build 2025-12-04T09:33:17.0436792Z VLLM_TEST_HUGGING_FACE_TOKEN=*** 2025-12-04T09:33:17.0437130Z HOSTNAME=77ce1fdc01d3 2025-12-04T09:33:17.0437853Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_4909f104-ffeb-4dc5-9e8c-4243af229733 2025-12-04T09:33:17.0438743Z GITHUB_ACTION=__run_3 2025-12-04T09:33:17.0439041Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2025-12-04T09:33:17.0439432Z GITHUB_RUN_NUMBER=67027 2025-12-04T09:33:17.0439972Z TEST_CONFIG=cpu_inductor_freezing_avx2_huggingface 2025-12-04T09:33:17.0440758Z GITHUB_REPOSITORY_OWNER_ID=21003710 2025-12-04T09:33:17.0441426Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2025-12-04T09:33:17.0442090Z SCCACHE_IDLE_TIMEOUT=0 2025-12-04T09:33:17.0442875Z SCRIBE_GRAPHQL_ACCESS_TOKEN=*** 2025-12-04T09:33:17.0443235Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot 2025-12-04T09:33:17.0443591Z GITHUB_REF_TYPE=branch 2025-12-04T09:33:17.0443940Z BASE_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:33:17.0444342Z XLA_CUDA= 2025-12-04T09:33:17.0444596Z NCCL_LIB_DIR=/usr/local/cuda/lib64/ 2025-12-04T09:33:17.0445137Z HUGGING_FACE_HUB_TOKEN=*** 2025-12-04T09:33:17.0445506Z *** 2025-12-04T09:33:17.0445779Z GITHUB_REPOSITORY_ID=65600975 2025-12-04T09:33:17.0446089Z GITHUB_ACTIONS=true 2025-12-04T09:33:17.0446434Z SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log 2025-12-04T09:33:17.0446893Z SHA1=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:33:17.0447327Z GITHUB_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:33:17.0448005Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/heads/main 2025-12-04T09:33:17.0448627Z UCC_HOME=/usr 2025-12-04T09:33:17.0448886Z TORCH_SERIALIZATION_DEBUG=1 2025-12-04T09:33:17.0449210Z VERBOSE_TEST_LOGS=False 2025-12-04T09:33:17.0449514Z GITHUB_REF=refs/heads/main 2025-12-04T09:33:17.0449805Z SHARD_NUMBER=1 2025-12-04T09:33:17.0450079Z GITHUB_REF_PROTECTED=true 2025-12-04T09:33:17.0450386Z HOME=/var/lib/jenkins 2025-12-04T09:33:17.0450696Z GITHUB_API_URL=https://api.github.com 2025-12-04T09:33:17.0451085Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2025-12-04T09:33:17.0451428Z UCX_COMMIT= 2025-12-04T09:33:17.0451685Z USE_SYSTEM_NCCL=1 2025-12-04T09:33:17.0451940Z NUM_TEST_SHARDS=1 2025-12-04T09:33:17.0452203Z UCX_HOME=/usr 2025-12-04T09:33:17.0452863Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_4909f104-ffeb-4dc5-9e8c-4243af229733 2025-12-04T09:33:17.0454006Z JOB_NAME=periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:33:17.0455130Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_4909f104-ffeb-4dc5-9e8c-4243af229733 2025-12-04T09:33:17.0456082Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2025-12-04T09:33:17.0456678Z GITHUB_EVENT_NAME=schedule 2025-12-04T09:33:17.0456971Z DASHBOARD_TAG= 2025-12-04T09:33:17.0457241Z GITHUB_RUN_ID=19923066595 2025-12-04T09:33:17.0457543Z INSTALLED_OPENBLAS= 2025-12-04T09:33:17.0458252Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_4909f104-ffeb-4dc5-9e8c-4243af229733 2025-12-04T09:33:17.0459279Z GITHUB_ACTOR=pytorchmergebot 2025-12-04T09:33:17.0459600Z PR_NUMBER= 2025-12-04T09:33:17.0459829Z DESIRED_CUDA= 2025-12-04T09:33:17.0460091Z GITHUB_RUN_ATTEMPT=1 2025-12-04T09:33:17.0460370Z VALGRIND=ON 2025-12-04T09:33:17.0460738Z ANACONDA_PYTHON_VERSION=3.10 2025-12-04T09:33:17.0461134Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2025-12-04T09:33:17.0461542Z TERM=vt100 2025-12-04T09:33:17.0461779Z INSTALLED_VISION=yes 2025-12-04T09:33:17.0462060Z BRANCH=main 2025-12-04T09:33:17.0462320Z SCCACHE_REGION=us-east-1 2025-12-04T09:33:17.0462622Z OPENSSL_ROOT_DIR=/opt/openssl 2025-12-04T09:33:17.0462957Z BUILD_AOT_INDUCTOR_TEST=yes 2025-12-04T09:33:17.0463276Z CUDA_PATH=/usr/local/cuda 2025-12-04T09:33:17.0463884Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2025-12-04T09:33:17.0464555Z GITHUB_SERVER_URL=https://github.com 2025-12-04T09:33:17.0464912Z UCC_COMMIT= 2025-12-04T09:33:17.0465164Z REENABLED_ISSUES= 2025-12-04T09:33:17.0465420Z DOCS=yes 2025-12-04T09:33:17.0465658Z SHLVL=1 2025-12-04T09:33:17.0465894Z MAX_JOBS=38 2025-12-04T09:33:17.0466136Z GITHUB_ACTOR_ID=97764156 2025-12-04T09:33:17.0466542Z GITHUB_WORKFLOW_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:33:17.0467311Z GITHUB_REF_NAME=main 2025-12-04T09:33:17.0467750Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2025-12-04T09:33:17.0468268Z GITHUB_JOB=test 2025-12-04T09:33:17.0468552Z NO_TEST_TIMEOUT=False 2025-12-04T09:33:17.0468863Z TD_DISTRIBUTED=False 2025-12-04T09:33:17.0469187Z GITHUB_REPOSITORY=pytorch/pytorch 2025-12-04T09:33:17.0469555Z GITHUB_RETENTION_DAYS=90 2025-12-04T09:33:17.0469859Z OPENSSL_DIR=/opt/openssl 2025-12-04T09:33:17.0470185Z GITHUB_ACTION_REPOSITORY= 2025-12-04T09:33:17.0471134Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-12-04T09:33:17.0472128Z GITHUB_BASE_REF= 2025-12-04T09:33:17.0472397Z INSTALLED_ACL= 2025-12-04T09:33:17.0472994Z ARTIFACTS_FILE_SUFFIX=test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305 2025-12-04T09:33:17.0473670Z CI=true 2025-12-04T09:33:17.0473925Z GITHUB_REPOSITORY_OWNER=pytorch 2025-12-04T09:33:17.0474320Z RUST_LOG=sccache::server=error 2025-12-04T09:33:17.0474656Z JOB_ID=57118563305 2025-12-04T09:33:17.0474919Z GITHUB_HEAD_REF= 2025-12-04T09:33:17.0475197Z GITHUB_ACTION_REF= 2025-12-04T09:33:17.0475548Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2025-12-04T09:33:17.0475966Z TEST_SHOWLOCALS=False 2025-12-04T09:33:17.0476292Z GITHUB_WORKFLOW=inductor-periodic 2025-12-04T09:33:17.0476665Z DEBIAN_FRONTEND=noninteractive 2025-12-04T09:33:17.0477626Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_4909f104-ffeb-4dc5-9e8c-4243af229733 2025-12-04T09:33:17.0478401Z NO_TD=False 2025-12-04T09:33:17.0478683Z SKIP_SCCACHE_INITIALIZATION=1 2025-12-04T09:33:17.0479121Z NCCL_INCLUDE_DIR=/usr/local/cuda/include/ 2025-12-04T09:33:17.0479487Z _=/usr/bin/env 2025-12-04T09:33:17.0479763Z + echo 'Testing pytorch' 2025-12-04T09:33:17.0480078Z Testing pytorch 2025-12-04T09:33:17.0480348Z + export LANG=C.UTF-8 2025-12-04T09:33:17.0480639Z + LANG=C.UTF-8 2025-12-04T09:33:17.0480926Z + PR_NUMBER= 2025-12-04T09:33:17.0481283Z + [[ cpu_inductor_freezing_avx2_huggingface == \d\e\f\a\u\l\t ]] 2025-12-04T09:33:17.0481890Z + [[ cpu_inductor_freezing_avx2_huggingface == \d\i\s\t\r\i\b\u\t\e\d ]] 2025-12-04T09:33:17.0482469Z + [[ cpu_inductor_freezing_avx2_huggingface == \s\l\o\w ]] 2025-12-04T09:33:17.0482999Z + [[ linux-jammy-py3.10-gcc11-build == *slow-gradcheck* ]] 2025-12-04T09:33:17.0483473Z + [[ linux-jammy-py3.10-gcc11-build == *cuda* ]] 2025-12-04T09:33:17.0483918Z + [[ linux-jammy-py3.10-gcc11-build == *rocm* ]] 2025-12-04T09:33:17.0484362Z + [[ linux-jammy-py3.10-gcc11-build == *xpu* ]] 2025-12-04T09:33:17.0484827Z + [[ cpu_inductor_freezing_avx2_huggingface == *crossref* ]] 2025-12-04T09:33:17.0485441Z + [[ linux-jammy-py3.10-gcc11-build == *rocm* ]] 2025-12-04T09:33:17.0485882Z + [[ linux-jammy-py3.10-gcc11-build == *xpu* ]] 2025-12-04T09:33:17.0486338Z + [[ linux-jammy-py3.10-gcc11-build != *-bazel-* ]] 2025-12-04T09:33:17.0486748Z + pip_install ninja==1.10.2 2025-12-04T09:33:17.0487264Z + pip_install_pkg='python3 -m pip install --progress-bar off' 2025-12-04T09:33:17.0487824Z + python3 -m pip install --progress-bar off ninja==1.10.2 2025-12-04T09:33:17.5652811Z Collecting ninja==1.10.2 2025-12-04T09:33:17.5928927Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2025-12-04T09:33:17.6061839Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2025-12-04T09:33:18.8445606Z Installing collected packages: ninja 2025-12-04T09:33:18.8446231Z Attempting uninstall: ninja 2025-12-04T09:33:18.8457159Z Found existing installation: ninja 1.11.1.4 2025-12-04T09:33:18.8485450Z Uninstalling ninja-1.11.1.4: 2025-12-04T09:33:18.8564585Z Successfully uninstalled ninja-1.11.1.4 2025-12-04T09:33:18.9340690Z Successfully installed ninja-1.10.2 2025-12-04T09:33:19.0372681Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-12-04T09:33:19.0375143Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2025-12-04T09:33:19.0376300Z + [[ linux-jammy-py3.10-gcc11-build == *aarch64* ]] 2025-12-04T09:33:19.0376761Z + [[ linux-jammy-py3.10-gcc11-build == *asan* ]] 2025-12-04T09:33:19.0377205Z + [[ linux-jammy-py3.10-gcc11-build == *-debug* ]] 2025-12-04T09:33:19.0377646Z + [[ linux-jammy-py3.10-gcc11-build != *-bazel-* ]] 2025-12-04T09:33:19.0378282Z + echo 'We are not in debug mode: linux-jammy-py3.10-gcc11-build. Expect the assertion to pass' 2025-12-04T09:33:19.0379085Z We are not in debug mode: linux-jammy-py3.10-gcc11-build. Expect the assertion to pass 2025-12-04T09:33:19.0379632Z + cd test 2025-12-04T09:33:19.0380025Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2025-12-04T09:33:19.4875105Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:33:19.4876885Z import pynvml # type: ignore[import] 2025-12-04T09:33:20.7878694Z + [[ cpu_inductor_freezing_avx2_huggingface == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2025-12-04T09:33:20.7879543Z + [[ cpu_inductor_freezing_avx2_huggingface == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2025-12-04T09:33:20.7880250Z + [[ cpu_inductor_freezing_avx2_huggingface == \l\e\g\a\c\y\_\n\v\i\d\i\a\_\d\r\i\v\e\r ]] 2025-12-04T09:33:20.7883874Z + DYNAMO_BENCHMARK_FLAGS=() 2025-12-04T09:33:20.7884995Z + [[ cpu_inductor_freezing_avx2_huggingface == *pr_time_benchmarks* ]] 2025-12-04T09:33:20.7885871Z + [[ cpu_inductor_freezing_avx2_huggingface == *dynamo_eager* ]] 2025-12-04T09:33:20.7886440Z + [[ cpu_inductor_freezing_avx2_huggingface == *aot_eager* ]] 2025-12-04T09:33:20.7887451Z + [[ cpu_inductor_freezing_avx2_huggingface == *aot_inductor* ]] 2025-12-04T09:33:20.7888058Z + [[ cpu_inductor_freezing_avx2_huggingface == *max_autotune_inductor* ]] 2025-12-04T09:33:20.7888620Z + [[ cpu_inductor_freezing_avx2_huggingface == *inductor* ]] 2025-12-04T09:33:20.7889118Z + [[ cpu_inductor_freezing_avx2_huggingface != *perf* ]] 2025-12-04T09:33:20.7889573Z + DYNAMO_BENCHMARK_FLAGS+=(--inductor) 2025-12-04T09:33:20.7889995Z + [[ cpu_inductor_freezing_avx2_huggingface == *dynamic* ]] 2025-12-04T09:33:20.7890482Z + [[ cpu_inductor_freezing_avx2_huggingface == *cpu* ]] 2025-12-04T09:33:20.7890913Z + DYNAMO_BENCHMARK_FLAGS+=(--device cpu) 2025-12-04T09:33:20.7932329Z + [[ linux-jammy-py3.10-gcc11-build == *libtorch* ]] 2025-12-04T09:33:20.7933367Z + [[ linux-jammy-py3.10-gcc11-build == *-bazel-* ]] 2025-12-04T09:33:20.7933776Z + cd test 2025-12-04T09:33:20.7934128Z + python -c 'import torch; print(torch.__config__.show())' 2025-12-04T09:33:21.2406877Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:33:21.2408761Z import pynvml # type: ignore[import] 2025-12-04T09:33:22.2292674Z PyTorch built with: 2025-12-04T09:33:22.2293244Z - GCC 11.4 2025-12-04T09:33:22.2293518Z - C++ Version: 201703 2025-12-04T09:33:22.2294194Z - Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications 2025-12-04T09:33:22.2295083Z - Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d) 2025-12-04T09:33:22.2295653Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2025-12-04T09:33:22.2296051Z - LAPACK is enabled (usually provided by MKL) 2025-12-04T09:33:22.2296452Z - NNPACK is enabled 2025-12-04T09:33:22.2296766Z - CPU capability usage: AVX2 2025-12-04T09:33:22.2303325Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, COMMIT_SHA=ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -DC10_NODEPRECATED -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -faligned-new -Werror -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, TORCH_VERSION=2.10.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, USE_XCCL=OFF, USE_XPU=OFF, 2025-12-04T09:33:22.2309429Z 2025-12-04T09:33:22.4930668Z + cd test 2025-12-04T09:33:22.4931435Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2025-12-04T09:33:22.9382329Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:33:22.9383956Z import pynvml # type: ignore[import] 2025-12-04T09:33:23.9338480Z ATen/Parallel: 2025-12-04T09:33:23.9339426Z at::get_num_threads() : 20 2025-12-04T09:33:23.9339862Z at::get_num_interop_threads() : 20 2025-12-04T09:33:23.9340243Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2025-12-04T09:33:23.9340587Z omp_get_max_threads() : 20 2025-12-04T09:33:23.9341739Z Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications 2025-12-04T09:33:23.9342484Z mkl_get_max_threads() : 20 2025-12-04T09:33:23.9342938Z Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d) 2025-12-04T09:33:23.9343494Z std::thread::hardware_concurrency() : 40 2025-12-04T09:33:23.9343882Z Environment variables: 2025-12-04T09:33:23.9344182Z OMP_NUM_THREADS : [not set] 2025-12-04T09:33:23.9344512Z MKL_NUM_THREADS : [not set] 2025-12-04T09:33:23.9344851Z ATen parallel backend: OpenMP 2025-12-04T09:33:23.9345080Z 2025-12-04T09:33:24.2046002Z + [[ cpu_inductor_freezing_avx2_huggingface == *numpy_2* ]] 2025-12-04T09:33:24.2046986Z + [[ linux-jammy-py3.10-gcc11-build == *aarch64* ]] 2025-12-04T09:33:24.2047488Z + [[ cpu_inductor_freezing_avx2_huggingface == *backward* ]] 2025-12-04T09:33:24.2048100Z + [[ cpu_inductor_freezing_avx2_huggingface == *libtorch_agnostic_targetting* ]] 2025-12-04T09:33:24.2048858Z + [[ cpu_inductor_freezing_avx2_huggingface == *xla* ]] 2025-12-04T09:33:24.2049323Z + [[ cpu_inductor_freezing_avx2_huggingface == *vllm* ]] 2025-12-04T09:33:24.2049833Z + [[ cpu_inductor_freezing_avx2_huggingface == *executorch* ]] 2025-12-04T09:33:24.2050407Z + [[ cpu_inductor_freezing_avx2_huggingface == \j\i\t\_\l\e\g\a\c\y ]] 2025-12-04T09:33:24.2051127Z + [[ cpu_inductor_freezing_avx2_huggingface == \q\u\a\n\t\i\z\a\t\i\o\n ]] 2025-12-04T09:33:24.2051693Z + [[ linux-jammy-py3.10-gcc11-build == *libtorch* ]] 2025-12-04T09:33:24.2052190Z + [[ cpu_inductor_freezing_avx2_huggingface == distributed ]] 2025-12-04T09:33:24.2052749Z + [[ cpu_inductor_freezing_avx2_huggingface == *operator_benchmark* ]] 2025-12-04T09:33:24.2063878Z + [[ cpu_inductor_freezing_avx2_huggingface == *operator_microbenchmark* ]] 2025-12-04T09:33:24.2064596Z + [[ cpu_inductor_freezing_avx2_huggingface == *attention_microbenchmark* ]] 2025-12-04T09:33:24.2065271Z + [[ cpu_inductor_freezing_avx2_huggingface == *inductor_distributed* ]] 2025-12-04T09:33:24.2065904Z + [[ cpu_inductor_freezing_avx2_huggingface == *inductor-halide* ]] 2025-12-04T09:33:24.2066839Z + [[ cpu_inductor_freezing_avx2_huggingface == *inductor-pallas* ]] 2025-12-04T09:33:24.2067471Z + [[ cpu_inductor_freezing_avx2_huggingface == *inductor-triton-cpu* ]] 2025-12-04T09:33:24.2068149Z + [[ cpu_inductor_freezing_avx2_huggingface == *inductor-micro-benchmark* ]] 2025-12-04T09:33:24.2068874Z + [[ cpu_inductor_freezing_avx2_huggingface == *aoti_cross_compile_for_windows* ]] 2025-12-04T09:33:24.2069520Z + [[ cpu_inductor_freezing_avx2_huggingface == *huggingface* ]] 2025-12-04T09:33:24.2069994Z + install_torchvision 2025-12-04T09:33:24.2070302Z + local orig_preload 2025-12-04T09:33:24.2070579Z + local commit 2025-12-04T09:33:24.2070893Z ++ get_pinned_commit vision 2025-12-04T09:33:24.2071248Z ++ cat .github/ci_commit_pins/vision.txt 2025-12-04T09:33:24.2071663Z + commit=617079d944b0e72632311c30ae2bbdf1168b901e 2025-12-04T09:33:24.2072071Z + orig_preload= 2025-12-04T09:33:24.2072349Z + '[' -n '' ']' 2025-12-04T09:33:24.2072679Z + [[ linux-jammy-py3.10-gcc11-build == *cuda* ]] 2025-12-04T09:33:24.2073484Z + pip_build_and_install git+https://github.com/pytorch/vision.git@617079d944b0e72632311c30ae2bbdf1168b901e dist/vision 2025-12-04T09:33:24.2074556Z + local build_target=git+https://github.com/pytorch/vision.git@617079d944b0e72632311c30ae2bbdf1168b901e 2025-12-04T09:33:24.2075250Z + local wheel_dir=dist/vision 2025-12-04T09:33:24.2075572Z + local found_whl=0 2025-12-04T09:33:24.2075876Z + for file in "${wheel_dir}"/*.whl 2025-12-04T09:33:24.2076420Z + [[ -f dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl ]] 2025-12-04T09:33:24.2076957Z + found_whl=1 2025-12-04T09:33:24.2077211Z + break 2025-12-04T09:33:24.2077456Z + '[' 1 == 0 ']' 2025-12-04T09:33:24.2077746Z + for file in "${wheel_dir}"/*.whl 2025-12-04T09:33:24.2078322Z + pip_install_whl dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl 2025-12-04T09:33:24.2079246Z + args=('dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl') 2025-12-04T09:33:24.2079783Z + local args 2025-12-04T09:33:24.2080229Z + [[ dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl == *\ * ]] 2025-12-04T09:33:24.2080805Z + for path in "${args[@]}" 2025-12-04T09:33:24.2081357Z + echo 'Installing dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl' 2025-12-04T09:33:24.2082150Z Installing dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl 2025-12-04T09:33:24.2083069Z + python3 -mpip install --no-index --no-deps dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl 2025-12-04T09:33:24.6425846Z Processing ./dist/vision/torchvision-0.25.0a0+617079d-cp310-cp310-linux_x86_64.whl 2025-12-04T09:33:24.6551212Z Installing collected packages: torchvision 2025-12-04T09:33:25.2237651Z Successfully installed torchvision-0.25.0a0+617079d 2025-12-04T09:33:25.2669899Z + '[' -n '' ']' 2025-12-04T09:33:25.2670228Z + id=0 2025-12-04T09:33:25.2670519Z + test_dynamo_benchmark huggingface 0 2025-12-04T09:33:25.2673006Z ++ pwd 2025-12-04T09:33:25.2674870Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2025-12-04T09:33:25.2675497Z + local suite=huggingface 2025-12-04T09:33:25.2675814Z + shift 2025-12-04T09:33:25.2676073Z + local shard_id=0 2025-12-04T09:33:25.2676331Z + shift 2025-12-04T09:33:25.2676573Z + extra_args=() 2025-12-04T09:33:25.2676848Z + local extra_args 2025-12-04T09:33:25.2677171Z + [[ linux-jammy-py3.10-gcc11-build == *cuda13* ]] 2025-12-04T09:33:25.2677704Z + [[ cpu_inductor_freezing_avx2_huggingface == *perf_compare* ]] 2025-12-04T09:33:25.2678252Z + [[ cpu_inductor_freezing_avx2_huggingface == *perf* ]] 2025-12-04T09:33:25.2678732Z + [[ cpu_inductor_freezing_avx2_huggingface == *cpu* ]] 2025-12-04T09:33:25.2679173Z + local dt=float32 2025-12-04T09:33:25.2679522Z + [[ cpu_inductor_freezing_avx2_huggingface == *amp* ]] 2025-12-04T09:33:25.2680026Z + [[ cpu_inductor_freezing_avx2_huggingface == *freezing* ]] 2025-12-04T09:33:25.2680704Z + test_single_dynamo_benchmark inference huggingface 0 --inference --float32 --freezing 2025-12-04T09:33:25.2681421Z ++ pwd 2025-12-04T09:33:25.2682449Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2025-12-04T09:33:25.2682973Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2025-12-04T09:33:25.2702507Z + local name=inference 2025-12-04T09:33:25.2702958Z + shift 2025-12-04T09:33:25.2703228Z + local suite=huggingface 2025-12-04T09:33:25.2703526Z + shift 2025-12-04T09:33:25.2703780Z + local shard_id=0 2025-12-04T09:33:25.2704060Z + shift 2025-12-04T09:33:25.2704300Z + partition_flags=() 2025-12-04T09:33:25.2704611Z + local partition_flags 2025-12-04T09:33:25.2704915Z + [[ -n 1 ]] 2025-12-04T09:33:25.2705175Z + [[ -n 0 ]] 2025-12-04T09:33:25.2705667Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2025-12-04T09:33:25.2706374Z + [[ cpu_inductor_freezing_avx2_huggingface == *perf_compare* ]] 2025-12-04T09:33:25.2707078Z + [[ cpu_inductor_freezing_avx2_huggingface == *perf* ]] 2025-12-04T09:33:25.2707595Z + [[ cpu_inductor_freezing_avx2_huggingface == *_avx2* ]] 2025-12-04T09:33:25.2708086Z + TEST_CONFIG=cpu_inductor_freezing_huggingface 2025-12-04T09:33:25.2708555Z + [[ cpu_inductor_freezing_huggingface == *_avx512* ]] 2025-12-04T09:33:25.2710185Z + python benchmarks/dynamo/huggingface.py --ci --accuracy --timing --explain --print-compilation-time --inductor --device cpu --inference --float32 --freezing --total-partitions 1 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_huggingface.csv 2025-12-04T09:33:26.3946595Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:33:26.3948802Z import pynvml # type: ignore[import] 2025-12-04T09:33:31.2861974Z 2025-12-04T09:33:31.2862692Z config.json: 0% 0.00/694 [00:00bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7509266Z 2025-12-04T09:35:37.7509463Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7510594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7511672Z layer_outputs = layer_module( 2025-12-04T09:35:37.7512400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7513174Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7514088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7515005Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7515898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7516789Z self_outputs = self.self( 2025-12-04T09:35:37.7517651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7518606Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7519697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7520905Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.7521969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.7522864Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.7523147Z 2025-12-04T09:35:37.7523345Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7524467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7525544Z layer_outputs = layer_module( 2025-12-04T09:35:37.7526258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7527033Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7527948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7528865Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7529741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7530632Z self_outputs = self.self( 2025-12-04T09:35:37.7531475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7532435Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7533508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7534997Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7535516Z 2025-12-04T09:35:37.7535729Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7536953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7538046Z layer_outputs = layer_module( 2025-12-04T09:35:37.7538778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7539557Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7540450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7541371Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7542276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7543189Z self_outputs = self.self( 2025-12-04T09:35:37.7544039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7545000Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7546086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7547517Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7548042Z 2025-12-04T09:35:37.7548245Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7549387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7550464Z layer_outputs = layer_module( 2025-12-04T09:35:37.7551193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7551954Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7552857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7553776Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7554655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7555547Z self_outputs = self.self( 2025-12-04T09:35:37.7556405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7557364Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7558439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7559718Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7560256Z 2025-12-04T09:35:37.7560410Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7560821Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7561203Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7561604Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7562062Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7563186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7564249Z layer_outputs = layer_module( 2025-12-04T09:35:37.7565121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7566013Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7567009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7567924Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7568890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7569788Z self_outputs = self.self( 2025-12-04T09:35:37.7570631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 535, in forward 2025-12-04T09:35:37.7571603Z diagonal_mask = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7572701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 833, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7573894Z self._mask_invalid_locations(diagonal_attention_scores, window_overlap) 2025-12-04T09:35:37.7575025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 763, in _mask_invalid_locations 2025-12-04T09:35:37.7576059Z ).where(beginning_mask.bool(), beginning_input) 2025-12-04T09:35:37.7576357Z 2025-12-04T09:35:37.7576514Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7576950Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7578069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7579133Z layer_outputs = layer_module( 2025-12-04T09:35:37.7579850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7580609Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7581506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7582416Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7583304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7584185Z self_outputs = self.self( 2025-12-04T09:35:37.7585035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.7586025Z attn_scores += diagonal_mask 2025-12-04T09:35:37.7586251Z 2025-12-04T09:35:37.7586436Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7587800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7588879Z layer_outputs = layer_module( 2025-12-04T09:35:37.7589604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7590355Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7591249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7592157Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7593046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7593923Z self_outputs = self.self( 2025-12-04T09:35:37.7594771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.7595758Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.7596019Z 2025-12-04T09:35:37.7596211Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7597397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7598464Z layer_outputs = layer_module( 2025-12-04T09:35:37.7599296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7600029Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7601532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7602454Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7603476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7604458Z self_outputs = self.self( 2025-12-04T09:35:37.7605312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7606311Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7607466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7608729Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.7609627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.7610355Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.7610660Z 2025-12-04T09:35:37.7610854Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7611986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7613158Z layer_outputs = layer_module( 2025-12-04T09:35:37.7613868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7614595Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7615464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7616346Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7617211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7618066Z self_outputs = self.self( 2025-12-04T09:35:37.7618894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7619851Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7620965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7622099Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.7623164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.7624140Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.7624801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.7625503Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.7625813Z 2025-12-04T09:35:37.7626001Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7627388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7628709Z layer_outputs = layer_module( 2025-12-04T09:35:37.7631128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7632025Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7632936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7633838Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7634732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7635626Z self_outputs = self.self( 2025-12-04T09:35:37.7636475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7637451Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7638604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7639936Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.7640359Z 2025-12-04T09:35:37.7640558Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7641633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7642741Z layer_outputs = layer_module( 2025-12-04T09:35:37.7643441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7644180Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7645060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7645950Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7646815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7647666Z self_outputs = self.self( 2025-12-04T09:35:37.7648491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7649441Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7650560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7651747Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.7652178Z 2025-12-04T09:35:37.7652368Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7653464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7654492Z layer_outputs = layer_module( 2025-12-04T09:35:37.7655178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7655926Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7656795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7657665Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7658528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7659487Z self_outputs = self.self( 2025-12-04T09:35:37.7660311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.7661456Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.7661967Z 2025-12-04T09:35:37.7662106Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7662496Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7662931Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7664007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7665036Z layer_outputs = layer_module( 2025-12-04T09:35:37.7665736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7666476Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7667632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.7668572Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.7669433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.7670267Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.7671159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.7672140Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.7673096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.7674059Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.7674856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.7675584Z return self.act(input) 2025-12-04T09:35:37.7675795Z 2025-12-04T09:35:37.7675939Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7676338Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7676785Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7677910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7678968Z layer_outputs = layer_module( 2025-12-04T09:35:37.7679786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7680526Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7681388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7682266Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7683134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7684014Z self_outputs = self.self( 2025-12-04T09:35:37.7684825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7685746Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7686797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7688045Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7688618Z 2025-12-04T09:35:37.7688758Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7689197Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7696041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7697118Z layer_outputs = layer_module( 2025-12-04T09:35:37.7697833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7698619Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7699525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7700426Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7701762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7702684Z self_outputs = self.self( 2025-12-04T09:35:37.7703538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7704482Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7705567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7706766Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.7707938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.7708832Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.7709113Z 2025-12-04T09:35:37.7709308Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7710434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7711505Z layer_outputs = layer_module( 2025-12-04T09:35:37.7712217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7712994Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7713897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7714803Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7715691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7716597Z self_outputs = self.self( 2025-12-04T09:35:37.7717454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7718397Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7719575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7720816Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7721320Z 2025-12-04T09:35:37.7721527Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7722598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7723625Z layer_outputs = layer_module( 2025-12-04T09:35:37.7724321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7725196Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7726051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7726928Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7727867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7728723Z self_outputs = self.self( 2025-12-04T09:35:37.7729549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7730477Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7731527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7732751Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7733273Z 2025-12-04T09:35:37.7733461Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7734554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7735580Z layer_outputs = layer_module( 2025-12-04T09:35:37.7736265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7737009Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7737879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7738758Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7739606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7740474Z self_outputs = self.self( 2025-12-04T09:35:37.7741300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7742214Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7743265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7744504Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7745001Z 2025-12-04T09:35:37.7745155Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7745530Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7745966Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7747130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7748439Z layer_outputs = layer_module( 2025-12-04T09:35:37.7749152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7749926Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7750835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7751748Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7757499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7758413Z self_outputs = self.self( 2025-12-04T09:35:37.7759264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.7760231Z attn_scores += diagonal_mask 2025-12-04T09:35:37.7760480Z 2025-12-04T09:35:37.7760673Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7761879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7762942Z layer_outputs = layer_module( 2025-12-04T09:35:37.7763648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7764526Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7765402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7766285Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7767133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7768004Z self_outputs = self.self( 2025-12-04T09:35:37.7768820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.7769695Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.7769961Z 2025-12-04T09:35:37.7770100Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7770933Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7772021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7773041Z layer_outputs = layer_module( 2025-12-04T09:35:37.7773740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7774487Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7775370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7776241Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7777118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7777984Z self_outputs = self.self( 2025-12-04T09:35:37.7778791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7779747Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7780861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7782099Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.7782958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.7783667Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.7783979Z 2025-12-04T09:35:37.7784169Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7785256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7786272Z layer_outputs = layer_module( 2025-12-04T09:35:37.7786973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7788065Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7788975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7789873Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7790854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7791758Z self_outputs = self.self( 2025-12-04T09:35:37.7792681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7793664Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7794825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7796014Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.7797108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.7798120Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.7798826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.7799557Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.7799864Z 2025-12-04T09:35:37.7800060Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7801623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7802696Z layer_outputs = layer_module( 2025-12-04T09:35:37.7803428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7804187Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7805093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7806012Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7806896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7807793Z self_outputs = self.self( 2025-12-04T09:35:37.7808651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7809636Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7810767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7811996Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.7812446Z 2025-12-04T09:35:37.7812639Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7813848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7814931Z layer_outputs = layer_module( 2025-12-04T09:35:37.7815930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7816701Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7817605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7818501Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7819398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7820368Z self_outputs = self.self( 2025-12-04T09:35:37.7821206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7822327Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7823551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7824774Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.7825214Z 2025-12-04T09:35:37.7825406Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7826526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7827677Z layer_outputs = layer_module( 2025-12-04T09:35:37.7828403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7829162Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7830072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7830989Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7831890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7832771Z self_outputs = self.self( 2025-12-04T09:35:37.7833622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.7834752Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.7835260Z 2025-12-04T09:35:37.7835417Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7835806Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7836255Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7837379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7838427Z layer_outputs = layer_module( 2025-12-04T09:35:37.7839248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7839990Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7840863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.7841738Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.7842570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.7843392Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.7844238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.7845193Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.7846131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.7847073Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.7847835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.7848538Z return self.act(input) 2025-12-04T09:35:37.7848755Z 2025-12-04T09:35:37.7848891Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7849282Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7849704Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7850793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7851892Z layer_outputs = layer_module( 2025-12-04T09:35:37.7852578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7853383Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7854260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7855144Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7856002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7856871Z self_outputs = self.self( 2025-12-04T09:35:37.7857698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7858619Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7859659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7860915Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7861418Z 2025-12-04T09:35:37.7861574Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7862003Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7863091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7864124Z layer_outputs = layer_module( 2025-12-04T09:35:37.7864825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7865557Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7866438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7867423Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7868492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7869384Z self_outputs = self.self( 2025-12-04T09:35:37.7870236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7871193Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7872277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7873471Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.7874549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.7875454Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.7875715Z 2025-12-04T09:35:37.7875913Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7877042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7878298Z layer_outputs = layer_module( 2025-12-04T09:35:37.7879027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7879782Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7880693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7881611Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7882660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7883546Z self_outputs = self.self( 2025-12-04T09:35:37.7884478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7885437Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7886516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7887786Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7888326Z 2025-12-04T09:35:37.7888520Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7889758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7890800Z layer_outputs = layer_module( 2025-12-04T09:35:37.7891496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7892249Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7893128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7894000Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7894859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7895724Z self_outputs = self.self( 2025-12-04T09:35:37.7896547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7897462Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7898509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7899748Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7900247Z 2025-12-04T09:35:37.7900448Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7902066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7903134Z layer_outputs = layer_module( 2025-12-04T09:35:37.7903861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7904636Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7905534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7906445Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7907445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7908338Z self_outputs = self.self( 2025-12-04T09:35:37.7909187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.7910133Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.7911223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.7912490Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.7913150Z 2025-12-04T09:35:37.7913298Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7913709Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7914168Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7915357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7916424Z layer_outputs = layer_module( 2025-12-04T09:35:37.7917144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7917902Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7918910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7919793Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7920656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7921518Z self_outputs = self.self( 2025-12-04T09:35:37.7922347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.7923216Z attn_scores += diagonal_mask 2025-12-04T09:35:37.7923440Z 2025-12-04T09:35:37.7923641Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7924717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7925746Z layer_outputs = layer_module( 2025-12-04T09:35:37.7926444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7927174Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7928053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7928938Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7929808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7930662Z self_outputs = self.self( 2025-12-04T09:35:37.7931488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.7932363Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.7932617Z 2025-12-04T09:35:37.7932770Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.7933192Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7934281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7935311Z layer_outputs = layer_module( 2025-12-04T09:35:37.7935999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7936745Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7937621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7938502Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7939357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7945299Z self_outputs = self.self( 2025-12-04T09:35:37.7946159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7947278Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7948508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7949851Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.7950748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.7951489Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.7951797Z 2025-12-04T09:35:37.7951992Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7953120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7954187Z layer_outputs = layer_module( 2025-12-04T09:35:37.7954898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7955672Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7956570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7957487Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7958364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7959363Z self_outputs = self.self( 2025-12-04T09:35:37.7960186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7961143Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7962246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7963402Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.7964469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.7965451Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.7966119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.7966833Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.7967131Z 2025-12-04T09:35:37.7967339Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7968416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7969453Z layer_outputs = layer_module( 2025-12-04T09:35:37.7970159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7970914Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7971778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7972670Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7973544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7974416Z self_outputs = self.self( 2025-12-04T09:35:37.7975226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7976187Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7977301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7978568Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.7978994Z 2025-12-04T09:35:37.7979181Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7980332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7981370Z layer_outputs = layer_module( 2025-12-04T09:35:37.7982058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7982802Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7983672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7984557Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7985412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7986282Z self_outputs = self.self( 2025-12-04T09:35:37.7987204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.7988351Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.7989486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.7990715Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.7991155Z 2025-12-04T09:35:37.7991365Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.7992496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.7993561Z layer_outputs = layer_module( 2025-12-04T09:35:37.7994290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.7995067Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.7995956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.7996876Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.7997771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.7998674Z self_outputs = self.self( 2025-12-04T09:35:37.7999508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.8000643Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.8001612Z 2025-12-04T09:35:37.8001758Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8022153Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8022636Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8023743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8024790Z layer_outputs = layer_module( 2025-12-04T09:35:37.8025505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8026263Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8027256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.8028359Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.8029403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.8030265Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.8031227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.8032217Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.8033185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.8034170Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.8034959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.8035703Z return self.act(input) 2025-12-04T09:35:37.8035915Z 2025-12-04T09:35:37.8036080Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8036470Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8036925Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8038064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8039131Z layer_outputs = layer_module( 2025-12-04T09:35:37.8039944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8040690Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8041569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8042448Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8043317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8044198Z self_outputs = self.self( 2025-12-04T09:35:37.8045025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8045942Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8046995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8048235Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8048741Z 2025-12-04T09:35:37.8048900Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8049330Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8050422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8051458Z layer_outputs = layer_module( 2025-12-04T09:35:37.8052155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8052909Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8053790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8054676Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8055528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8056403Z self_outputs = self.self( 2025-12-04T09:35:37.8057228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8058149Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8059257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8060475Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.8061513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.8062394Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.8062646Z 2025-12-04T09:35:37.8062836Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8063927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8065047Z layer_outputs = layer_module( 2025-12-04T09:35:37.8066072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8066849Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8067915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8068837Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8069788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8070699Z self_outputs = self.self( 2025-12-04T09:35:37.8071553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8072506Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8073579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8074872Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8075407Z 2025-12-04T09:35:37.8075602Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8076736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8077787Z layer_outputs = layer_module( 2025-12-04T09:35:37.8078513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8079374Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8080225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8081096Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8081958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8082825Z self_outputs = self.self( 2025-12-04T09:35:37.8083630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8084542Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8085578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8086803Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8087304Z 2025-12-04T09:35:37.8087486Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8088564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8089702Z layer_outputs = layer_module( 2025-12-04T09:35:37.8090392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8091192Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8092051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8092929Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8093778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8094618Z self_outputs = self.self( 2025-12-04T09:35:37.8095437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8096363Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8097396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8098617Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8099119Z 2025-12-04T09:35:37.8099260Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8099638Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8100045Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8101687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8102746Z layer_outputs = layer_module( 2025-12-04T09:35:37.8103460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8104215Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8105104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8106009Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8106889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8107897Z self_outputs = self.self( 2025-12-04T09:35:37.8108739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.8109608Z attn_scores += diagonal_mask 2025-12-04T09:35:37.8109837Z 2025-12-04T09:35:37.8110025Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8111137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8112198Z layer_outputs = layer_module( 2025-12-04T09:35:37.8112915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8113664Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8114558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8115462Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8116334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8117223Z self_outputs = self.self( 2025-12-04T09:35:37.8118049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.8119043Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.8119426Z 2025-12-04T09:35:37.8119561Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8119999Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8121152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8122176Z layer_outputs = layer_module( 2025-12-04T09:35:37.8122865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8123598Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8124466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8125326Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8126172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8127037Z self_outputs = self.self( 2025-12-04T09:35:37.8128324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8129295Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8130440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8131713Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.8132669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8133379Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8133695Z 2025-12-04T09:35:37.8133884Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8135017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8136058Z layer_outputs = layer_module( 2025-12-04T09:35:37.8136756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8137504Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8138396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8139275Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8140248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8141107Z self_outputs = self.self( 2025-12-04T09:35:37.8141912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8142849Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8143952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8145096Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.8146138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.8147192Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.8148048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8148762Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8149065Z 2025-12-04T09:35:37.8149253Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8150451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8151509Z layer_outputs = layer_module( 2025-12-04T09:35:37.8152268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8153024Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8153917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8154804Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8155682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8156564Z self_outputs = self.self( 2025-12-04T09:35:37.8157410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8158377Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8159667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8160852Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8161271Z 2025-12-04T09:35:37.8161461Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8162525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8163532Z layer_outputs = layer_module( 2025-12-04T09:35:37.8164226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8164957Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8165811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8166676Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8167533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8168375Z self_outputs = self.self( 2025-12-04T09:35:37.8169182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8170116Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8171209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8172375Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8172801Z 2025-12-04T09:35:37.8172980Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8174052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8175065Z layer_outputs = layer_module( 2025-12-04T09:35:37.8175735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8176458Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8177319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8178187Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8179027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8179934Z self_outputs = self.self( 2025-12-04T09:35:37.8180740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.8181877Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.8182361Z 2025-12-04T09:35:37.8182496Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8182870Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8183297Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8184364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8185373Z layer_outputs = layer_module( 2025-12-04T09:35:37.8186050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8186784Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8187906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.8188815Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.8189718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.8195608Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.8196474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.8197432Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.8198374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.8199330Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.8200106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.8201250Z return self.act(input) 2025-12-04T09:35:37.8201463Z 2025-12-04T09:35:37.8201619Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8201997Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8202421Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8203535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8204576Z layer_outputs = layer_module( 2025-12-04T09:35:37.8205271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8206017Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8206904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8207782Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8208658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8209529Z self_outputs = self.self( 2025-12-04T09:35:37.8210364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8211288Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8212354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8213697Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8214339Z 2025-12-04T09:35:37.8214484Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8214898Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8216053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8217072Z layer_outputs = layer_module( 2025-12-04T09:35:37.8217747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8218481Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8219344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8220214Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8221057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8221909Z self_outputs = self.self( 2025-12-04T09:35:37.8222707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8223613Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8224638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8225789Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.8226814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.8227938Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.8228261Z 2025-12-04T09:35:37.8228455Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8229572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8230624Z layer_outputs = layer_module( 2025-12-04T09:35:37.8231328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8232080Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8232962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8233854Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8234722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8235594Z self_outputs = self.self( 2025-12-04T09:35:37.8236421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8237355Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8238414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8239764Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8240257Z 2025-12-04T09:35:37.8240445Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8241512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8242520Z layer_outputs = layer_module( 2025-12-04T09:35:37.8243201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8243989Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8244834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8245696Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8246631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8247481Z self_outputs = self.self( 2025-12-04T09:35:37.8248272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8249168Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8250206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8251439Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8251940Z 2025-12-04T09:35:37.8252191Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8258382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8259435Z layer_outputs = layer_module( 2025-12-04T09:35:37.8260145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8260887Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8261766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8262662Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8263531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8264413Z self_outputs = self.self( 2025-12-04T09:35:37.8265340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8266255Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8267393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8268822Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8269341Z 2025-12-04T09:35:37.8269478Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8269868Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8270295Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8271405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8272467Z layer_outputs = layer_module( 2025-12-04T09:35:37.8273173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8273935Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8274816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8275703Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8276571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8277438Z self_outputs = self.self( 2025-12-04T09:35:37.8278265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.8279331Z attn_scores += diagonal_mask 2025-12-04T09:35:37.8279545Z 2025-12-04T09:35:37.8279726Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8280861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8281881Z layer_outputs = layer_module( 2025-12-04T09:35:37.8282556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8283271Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8284127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8284986Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8285817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8286675Z self_outputs = self.self( 2025-12-04T09:35:37.8287475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.8288343Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.8288591Z 2025-12-04T09:35:37.8288727Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8289141Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8290212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8291219Z layer_outputs = layer_module( 2025-12-04T09:35:37.8291894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8292620Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8293469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8294324Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8295179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8296045Z self_outputs = self.self( 2025-12-04T09:35:37.8296866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8297807Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8298925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8300160Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.8301581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8302309Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8302634Z 2025-12-04T09:35:37.8302829Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8303957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8305009Z layer_outputs = layer_module( 2025-12-04T09:35:37.8305728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8306495Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8307485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8308378Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8309394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8310295Z self_outputs = self.self( 2025-12-04T09:35:37.8311239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8312220Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8313371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8314602Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.8315818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.8316812Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.8317515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8318244Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8318551Z 2025-12-04T09:35:37.8318749Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8319935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8321002Z layer_outputs = layer_module( 2025-12-04T09:35:37.8321724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8322481Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8323385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8324301Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8325190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8326069Z self_outputs = self.self( 2025-12-04T09:35:37.8327034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8327990Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8329090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8330289Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8330732Z 2025-12-04T09:35:37.8330921Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8332009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8333046Z layer_outputs = layer_module( 2025-12-04T09:35:37.8333738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8334488Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8335363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8336234Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8337103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8337967Z self_outputs = self.self( 2025-12-04T09:35:37.8338789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8339805Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8340912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8342148Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8342573Z 2025-12-04T09:35:37.8342770Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8343834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8344856Z layer_outputs = layer_module( 2025-12-04T09:35:37.8345544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8346282Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8347231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8348303Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8349198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8350073Z self_outputs = self.self( 2025-12-04T09:35:37.8350910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.8352030Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.8352532Z 2025-12-04T09:35:37.8352687Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8353066Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8353509Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8354631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8355684Z layer_outputs = layer_module( 2025-12-04T09:35:37.8356388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8357149Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8358038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.8358940Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.8359875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.8360682Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.8361533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.8362460Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.8363382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.8364323Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.8365083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.8365763Z return self.act(input) 2025-12-04T09:35:37.8365966Z 2025-12-04T09:35:37.8366100Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8366477Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8366886Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8367962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8369049Z layer_outputs = layer_module( 2025-12-04T09:35:37.8369740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8370521Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8371384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8372251Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8373095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8373953Z self_outputs = self.self( 2025-12-04T09:35:37.8374761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8375669Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8376697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8378304Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8378829Z 2025-12-04T09:35:37.8378970Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8379419Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8380527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8381587Z layer_outputs = layer_module( 2025-12-04T09:35:37.8382366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8383128Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8384013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8384907Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8385796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8386669Z self_outputs = self.self( 2025-12-04T09:35:37.8387645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8388586Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8389653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8390830Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.8391886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.8392773Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.8393027Z 2025-12-04T09:35:37.8393228Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8394331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8395377Z layer_outputs = layer_module( 2025-12-04T09:35:37.8396094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8396848Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8397735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8398640Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8399693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8400547Z self_outputs = self.self( 2025-12-04T09:35:37.8402012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8402967Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8404040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8405288Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8405803Z 2025-12-04T09:35:37.8405989Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8407103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8408171Z layer_outputs = layer_module( 2025-12-04T09:35:37.8408869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8409628Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8410515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8411417Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8412287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8413174Z self_outputs = self.self( 2025-12-04T09:35:37.8414107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8415013Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8416054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8417290Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8417788Z 2025-12-04T09:35:37.8417979Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8419045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8420065Z layer_outputs = layer_module( 2025-12-04T09:35:37.8420762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8421500Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8422354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8423221Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8424075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8424939Z self_outputs = self.self( 2025-12-04T09:35:37.8425739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8426655Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8427952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8429209Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8429822Z 2025-12-04T09:35:37.8429963Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8430356Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8430795Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8431673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8431799Z layer_outputs = layer_module( 2025-12-04T09:35:37.8432296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8432434Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8433077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8433206Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8433836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8433973Z self_outputs = self.self( 2025-12-04T09:35:37.8434600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.8434722Z attn_scores += diagonal_mask 2025-12-04T09:35:37.8434750Z 2025-12-04T09:35:37.8434938Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8435732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8435870Z layer_outputs = layer_module( 2025-12-04T09:35:37.8436354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8436489Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8437134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8437270Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8437910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8438030Z self_outputs = self.self( 2025-12-04T09:35:37.8438663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.8438816Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.8438831Z 2025-12-04T09:35:37.8438970Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8439172Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8444846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8444980Z layer_outputs = layer_module( 2025-12-04T09:35:37.8445471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8445611Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8446239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8446385Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8447009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8447138Z self_outputs = self.self( 2025-12-04T09:35:37.8447762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8447971Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8448857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8449196Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.8449669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8449843Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8449859Z 2025-12-04T09:35:37.8450045Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8450851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8450975Z layer_outputs = layer_module( 2025-12-04T09:35:37.8451465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8451605Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8452337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8452475Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8453086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8453204Z self_outputs = self.self( 2025-12-04T09:35:37.8453815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8454024Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8454801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8455049Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.8455735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.8455891Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.8456285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8456459Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8456474Z 2025-12-04T09:35:37.8456655Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8457425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8457555Z layer_outputs = layer_module( 2025-12-04T09:35:37.8458023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8458167Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8458776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8458908Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8459528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8459644Z self_outputs = self.self( 2025-12-04T09:35:37.8460247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8460462Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8461228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8461574Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8461589Z 2025-12-04T09:35:37.8461772Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8462594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8462725Z layer_outputs = layer_module( 2025-12-04T09:35:37.8463194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8463338Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8463940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8464068Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8464691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8464815Z self_outputs = self.self( 2025-12-04T09:35:37.8465430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8465636Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8466406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8466698Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8466713Z 2025-12-04T09:35:37.8466894Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8467965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8468100Z layer_outputs = layer_module( 2025-12-04T09:35:37.8468583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8468737Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8469371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8469500Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8470148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8470271Z self_outputs = self.self( 2025-12-04T09:35:37.8470907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.8471271Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.8471291Z 2025-12-04T09:35:37.8471430Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8471577Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8471765Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8472568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8472692Z layer_outputs = layer_module( 2025-12-04T09:35:37.8473175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8473327Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8473958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.8474104Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.8474748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.8474882Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.8475579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.8475781Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.8476408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.8476612Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.8477068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.8477196Z return self.act(input) 2025-12-04T09:35:37.8477212Z 2025-12-04T09:35:37.8477350Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8477491Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8477681Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8478487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8478614Z layer_outputs = layer_module( 2025-12-04T09:35:37.8479106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8479243Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8479978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8480107Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8480718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8480850Z self_outputs = self.self( 2025-12-04T09:35:37.8481452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8481643Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8482384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8482739Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8482754Z 2025-12-04T09:35:37.8482900Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8483080Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8483853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8483975Z layer_outputs = layer_module( 2025-12-04T09:35:37.8484438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8484584Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8485200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8485330Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8485949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8486064Z self_outputs = self.self( 2025-12-04T09:35:37.8486676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8486850Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8487667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8487961Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.8488619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.8488761Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.8488776Z 2025-12-04T09:35:37.8488960Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8489728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8489861Z layer_outputs = layer_module( 2025-12-04T09:35:37.8490325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8490471Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8491084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8491217Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8491839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8491956Z self_outputs = self.self( 2025-12-04T09:35:37.8492558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8492741Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8493480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8493847Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8493861Z 2025-12-04T09:35:37.8494045Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8494817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8494949Z layer_outputs = layer_module( 2025-12-04T09:35:37.8495415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8495554Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8496169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8496299Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8496909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8497032Z self_outputs = self.self( 2025-12-04T09:35:37.8497649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8497822Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8498570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8498930Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8498945Z 2025-12-04T09:35:37.8499127Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8499910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8500084Z layer_outputs = layer_module( 2025-12-04T09:35:37.8500547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8500697Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8502052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8502220Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8507509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8507637Z self_outputs = self.self( 2025-12-04T09:35:37.8508277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8508459Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8509229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8509615Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8509631Z 2025-12-04T09:35:37.8509774Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8509926Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8510115Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8510907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8511040Z layer_outputs = layer_module( 2025-12-04T09:35:37.8511528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8511678Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8512314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8512446Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8513095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8513212Z self_outputs = self.self( 2025-12-04T09:35:37.8513839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.8513973Z attn_scores += diagonal_mask 2025-12-04T09:35:37.8513989Z 2025-12-04T09:35:37.8514177Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8514980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8515111Z layer_outputs = layer_module( 2025-12-04T09:35:37.8515594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8515734Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8516367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8516513Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8517139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8517253Z self_outputs = self.self( 2025-12-04T09:35:37.8517874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.8518003Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.8518018Z 2025-12-04T09:35:37.8518247Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8518447Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8519400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8519539Z layer_outputs = layer_module( 2025-12-04T09:35:37.8520009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8520139Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8520757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8520889Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8521497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8521618Z self_outputs = self.self( 2025-12-04T09:35:37.8522218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8522443Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8523214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8523556Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.8523948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8524113Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8524128Z 2025-12-04T09:35:37.8524328Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8525097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8525219Z layer_outputs = layer_module( 2025-12-04T09:35:37.8525707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8525843Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8526458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8526587Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8527195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8527325Z self_outputs = self.self( 2025-12-04T09:35:37.8527935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8528153Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8528929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8529172Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.8529871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.8530025Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.8530431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8530602Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8530617Z 2025-12-04T09:35:37.8530801Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8531636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8531760Z layer_outputs = layer_module( 2025-12-04T09:35:37.8532286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8532433Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8533045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8533181Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8533791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8533911Z self_outputs = self.self( 2025-12-04T09:35:37.8534530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8534745Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8535527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8535811Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8535826Z 2025-12-04T09:35:37.8536014Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8536794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8536919Z layer_outputs = layer_module( 2025-12-04T09:35:37.8537401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8537541Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8538157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8538299Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8538914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8539045Z self_outputs = self.self( 2025-12-04T09:35:37.8539652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8539853Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8540627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8540913Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8540932Z 2025-12-04T09:35:37.8541110Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8541893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8542014Z layer_outputs = layer_module( 2025-12-04T09:35:37.8542495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8542629Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8543237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8543380Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8543991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8544190Z self_outputs = self.self( 2025-12-04T09:35:37.8544796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.8545201Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.8545217Z 2025-12-04T09:35:37.8545369Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8545505Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8545700Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8546472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8546593Z layer_outputs = layer_module( 2025-12-04T09:35:37.8547198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8547345Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8548141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.8548306Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.8548877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.8549022Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.8549663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.8549861Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.8550498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.8550706Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.8551175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.8551294Z return self.act(input) 2025-12-04T09:35:37.8551310Z 2025-12-04T09:35:37.8551455Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8551604Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8551792Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8552597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8552732Z layer_outputs = layer_module( 2025-12-04T09:35:37.8553223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8553375Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8554006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8554140Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8554783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8554905Z self_outputs = self.self( 2025-12-04T09:35:37.8555544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8555721Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8556486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8556870Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8556950Z 2025-12-04T09:35:37.8557087Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8557280Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8558147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8558271Z layer_outputs = layer_module( 2025-12-04T09:35:37.8558768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8558904Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8559640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8559784Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8560388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8560524Z self_outputs = self.self( 2025-12-04T09:35:37.8561129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8561307Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8562064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8562348Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.8562957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.8563091Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.8563106Z 2025-12-04T09:35:37.8563285Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8564069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8564192Z layer_outputs = layer_module( 2025-12-04T09:35:37.8564750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8564891Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8565810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8565957Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8566594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8566718Z self_outputs = self.self( 2025-12-04T09:35:37.8567364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8567553Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8568330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8568701Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8568717Z 2025-12-04T09:35:37.8568910Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8569780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8569911Z layer_outputs = layer_module( 2025-12-04T09:35:37.8570405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8570543Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8571279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8571418Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8572106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8572239Z self_outputs = self.self( 2025-12-04T09:35:37.8572864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8573048Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8573824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8574193Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8574214Z 2025-12-04T09:35:37.8574417Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8575217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8575348Z layer_outputs = layer_module( 2025-12-04T09:35:37.8575842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8575982Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8576615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8576760Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8577388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8577647Z self_outputs = self.self( 2025-12-04T09:35:37.8578251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8578428Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8579187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8579544Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8579559Z 2025-12-04T09:35:37.8579710Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8579843Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8580024Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8580815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8580940Z layer_outputs = layer_module( 2025-12-04T09:35:37.8581408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8581561Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8582171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8582316Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8582921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8583036Z self_outputs = self.self( 2025-12-04T09:35:37.8583653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.8583772Z attn_scores += diagonal_mask 2025-12-04T09:35:37.8583840Z 2025-12-04T09:35:37.8584039Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8584885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8585006Z layer_outputs = layer_module( 2025-12-04T09:35:37.8585489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8585625Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8586242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8586372Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8586987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8587210Z self_outputs = self.self( 2025-12-04T09:35:37.8588128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.8588262Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.8588282Z 2025-12-04T09:35:37.8588438Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8588626Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8589428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8589552Z layer_outputs = layer_module( 2025-12-04T09:35:37.8590034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8590182Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8590816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8590960Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8591590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8591713Z self_outputs = self.self( 2025-12-04T09:35:37.8592347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8592560Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8593349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8593696Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.8594108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8594296Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8594312Z 2025-12-04T09:35:37.8594502Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8595304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8595435Z layer_outputs = layer_module( 2025-12-04T09:35:37.8595918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8596064Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8596691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8596826Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8597551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8597676Z self_outputs = self.self( 2025-12-04T09:35:37.8598836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8599052Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8599850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8600111Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.8601263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.8601435Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.8601866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8602043Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8602059Z 2025-12-04T09:35:37.8602266Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8603056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8603184Z layer_outputs = layer_module( 2025-12-04T09:35:37.8603686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8603822Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8604472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8604603Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8605239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8605372Z self_outputs = self.self( 2025-12-04T09:35:37.8605999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8606228Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8607019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8607310Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8607326Z 2025-12-04T09:35:37.8607530Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8608321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8608466Z layer_outputs = layer_module( 2025-12-04T09:35:37.8608957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8609092Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8609739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8609872Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8610495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8610633Z self_outputs = self.self( 2025-12-04T09:35:37.8611256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8611612Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8612407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8612774Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8612790Z 2025-12-04T09:35:37.8612998Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8613881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8614017Z layer_outputs = layer_module( 2025-12-04T09:35:37.8614488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8614624Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8615256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8615388Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8616015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8616133Z self_outputs = self.self( 2025-12-04T09:35:37.8616739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.8617103Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.8617118Z 2025-12-04T09:35:37.8617258Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8617391Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8617590Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8618365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8618500Z layer_outputs = layer_module( 2025-12-04T09:35:37.8618972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8619108Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8619728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.8619871Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.8620437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.8620564Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.8621179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.8621385Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.8621994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.8622194Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.8622645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.8622764Z return self.act(input) 2025-12-04T09:35:37.8622779Z 2025-12-04T09:35:37.8622928Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8623062Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8623245Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8624031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8624211Z layer_outputs = layer_module( 2025-12-04T09:35:37.8624692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8624827Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8625488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8625632Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8626234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8626350Z self_outputs = self.self( 2025-12-04T09:35:37.8626982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8627385Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8628449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8628818Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8628838Z 2025-12-04T09:35:37.8628978Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8629180Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8629977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8630115Z layer_outputs = layer_module( 2025-12-04T09:35:37.8630597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8640608Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8641342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8641492Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8642121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8642252Z self_outputs = self.self( 2025-12-04T09:35:37.8642856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8643044Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8643786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8644072Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.8644683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.8644818Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.8644835Z 2025-12-04T09:35:37.8645035Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8645817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8645940Z layer_outputs = layer_module( 2025-12-04T09:35:37.8646418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8646551Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8647180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8647309Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8648038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8648169Z self_outputs = self.self( 2025-12-04T09:35:37.8648848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8649027Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8649778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8650133Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8650149Z 2025-12-04T09:35:37.8650345Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8651117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8651245Z layer_outputs = layer_module( 2025-12-04T09:35:37.8651726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8651865Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8652489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8652622Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8653231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8653367Z self_outputs = self.self( 2025-12-04T09:35:37.8653972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8654162Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8654906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8655269Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8655284Z 2025-12-04T09:35:37.8655476Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8656245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8656371Z layer_outputs = layer_module( 2025-12-04T09:35:37.8656839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8656973Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8657594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8657731Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8658345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8658471Z self_outputs = self.self( 2025-12-04T09:35:37.8659074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8659255Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8659998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8660356Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8660371Z 2025-12-04T09:35:37.8660576Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8660710Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8660903Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8661731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8661854Z layer_outputs = layer_module( 2025-12-04T09:35:37.8662332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8662465Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8663076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8663210Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8663819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8663951Z self_outputs = self.self( 2025-12-04T09:35:37.8664552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.8664675Z attn_scores += diagonal_mask 2025-12-04T09:35:37.8664690Z 2025-12-04T09:35:37.8664883Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8665648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8665777Z layer_outputs = layer_module( 2025-12-04T09:35:37.8666244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8666378Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8667105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8667264Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8668068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8668186Z self_outputs = self.self( 2025-12-04T09:35:37.8668812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.8668960Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.8668975Z 2025-12-04T09:35:37.8669111Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8669301Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8670103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8670230Z layer_outputs = layer_module( 2025-12-04T09:35:37.8670726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8670862Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8671493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8671640Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8672265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8672400Z self_outputs = self.self( 2025-12-04T09:35:37.8673017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8673229Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8674102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8674438Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.8674905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8675093Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8675109Z 2025-12-04T09:35:37.8675295Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8676103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8676229Z layer_outputs = layer_module( 2025-12-04T09:35:37.8676709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8676860Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8677489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8677637Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8678263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8678382Z self_outputs = self.self( 2025-12-04T09:35:37.8679016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8679229Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8680110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8680354Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.8681032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.8681196Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.8681592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8681756Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8681783Z 2025-12-04T09:35:37.8681965Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8682743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8682873Z layer_outputs = layer_module( 2025-12-04T09:35:37.8683342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8683482Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8684103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8684237Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8684856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8684974Z self_outputs = self.self( 2025-12-04T09:35:37.8685575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8685786Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8686556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8686905Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8686921Z 2025-12-04T09:35:37.8687106Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8687932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8688065Z layer_outputs = layer_module( 2025-12-04T09:35:37.8688534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8688673Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8689279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8689407Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8695445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8695579Z self_outputs = self.self( 2025-12-04T09:35:37.8696211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8696436Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8697233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8697534Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8697551Z 2025-12-04T09:35:37.8697739Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8698538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8698674Z layer_outputs = layer_module( 2025-12-04T09:35:37.8699154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8699297Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8699931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8700061Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8700698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8701242Z self_outputs = self.self( 2025-12-04T09:35:37.8701888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.8702248Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.8702270Z 2025-12-04T09:35:37.8702412Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8702568Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8702754Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8703556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8703689Z layer_outputs = layer_module( 2025-12-04T09:35:37.8704171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8704314Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8704938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.8705082Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.8705811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.8705940Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.8706655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.8706850Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.8707574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.8707789Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.8708252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.8708370Z return self.act(input) 2025-12-04T09:35:37.8708395Z 2025-12-04T09:35:37.8708531Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8708671Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8708866Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8709668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8709791Z layer_outputs = layer_module( 2025-12-04T09:35:37.8710277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8710415Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8711052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8711186Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8711813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8711944Z self_outputs = self.self( 2025-12-04T09:35:37.8712564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8712747Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8713515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8713880Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8713896Z 2025-12-04T09:35:37.8714038Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8714221Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8715013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8715146Z layer_outputs = layer_module( 2025-12-04T09:35:37.8715629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8715771Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8716407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8716541Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8717177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8717295Z self_outputs = self.self( 2025-12-04T09:35:37.8717923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8718103Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8719043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8719335Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.8719983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.8720110Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.8720134Z 2025-12-04T09:35:37.8720314Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8721085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8721214Z layer_outputs = layer_module( 2025-12-04T09:35:37.8721679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8721815Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8722437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8722572Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8723199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8723317Z self_outputs = self.self( 2025-12-04T09:35:37.8723918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8724103Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8724841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8725208Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8725223Z 2025-12-04T09:35:37.8725403Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8726173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8726301Z layer_outputs = layer_module( 2025-12-04T09:35:37.8726764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8726902Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8727510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8727637Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8728257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8728381Z self_outputs = self.self( 2025-12-04T09:35:37.8728982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8729174Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8729917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8730279Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8730293Z 2025-12-04T09:35:37.8730479Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8731248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8731429Z layer_outputs = layer_module( 2025-12-04T09:35:37.8731898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8732038Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8732723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8732855Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8733466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8733580Z self_outputs = self.self( 2025-12-04T09:35:37.8734193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8734365Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8735112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8735473Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8735494Z 2025-12-04T09:35:37.8735631Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8735771Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8735955Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8736722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8736848Z layer_outputs = layer_module( 2025-12-04T09:35:37.8737313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8737445Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8738066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8738193Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8738813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8738927Z self_outputs = self.self( 2025-12-04T09:35:37.8739531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.8739660Z attn_scores += diagonal_mask 2025-12-04T09:35:37.8739676Z 2025-12-04T09:35:37.8739858Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8740634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8740754Z layer_outputs = layer_module( 2025-12-04T09:35:37.8741216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8741360Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8741975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8742100Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8742710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8742823Z self_outputs = self.self( 2025-12-04T09:35:37.8743429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.8743557Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.8743572Z 2025-12-04T09:35:37.8743761Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8743951Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8744772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8744896Z layer_outputs = layer_module( 2025-12-04T09:35:37.8745358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8745488Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8746108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8746232Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8746837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8746966Z self_outputs = self.self( 2025-12-04T09:35:37.8747833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8748058Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8748854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8749187Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.8749607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8749777Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8749793Z 2025-12-04T09:35:37.8749977Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8750784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8750913Z layer_outputs = layer_module( 2025-12-04T09:35:37.8751396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8751542Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8752250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8752391Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8757911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8758030Z self_outputs = self.self( 2025-12-04T09:35:37.8758662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8758883Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8759678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8759930Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.8760637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.8760800Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.8761206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8761376Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8761401Z 2025-12-04T09:35:37.8761587Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8762478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8762609Z layer_outputs = layer_module( 2025-12-04T09:35:37.8763147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8763282Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8763916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8764047Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8764796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8764911Z self_outputs = self.self( 2025-12-04T09:35:37.8765518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8765737Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8766508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8766798Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8766813Z 2025-12-04T09:35:37.8766995Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8767760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8767893Z layer_outputs = layer_module( 2025-12-04T09:35:37.8768360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8768500Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8769111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8769236Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8769854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8769970Z self_outputs = self.self( 2025-12-04T09:35:37.8770572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8770788Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8771545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8771833Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8771852Z 2025-12-04T09:35:37.8772030Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8772801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8772925Z layer_outputs = layer_module( 2025-12-04T09:35:37.8773386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8773519Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8774127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8774249Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8774861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8775028Z self_outputs = self.self( 2025-12-04T09:35:37.8775635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.8776039Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.8776055Z 2025-12-04T09:35:37.8776188Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8776322Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8776500Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8777272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8777390Z layer_outputs = layer_module( 2025-12-04T09:35:37.8777851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8777990Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8778600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.8778742Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.8779305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.8779429Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.8780053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.8780242Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.8780841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.8781037Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.8781484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.8781603Z return self.act(input) 2025-12-04T09:35:37.8781618Z 2025-12-04T09:35:37.8781750Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8781876Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8782064Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8782836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8782951Z layer_outputs = layer_module( 2025-12-04T09:35:37.8783428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8783557Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8784168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8784296Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8784904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8785021Z self_outputs = self.self( 2025-12-04T09:35:37.8785627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8785802Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8786544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8786897Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8786911Z 2025-12-04T09:35:37.8787214Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8787562Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8788415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8788544Z layer_outputs = layer_module( 2025-12-04T09:35:37.8789022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8789161Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8789785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8789913Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8790546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8790667Z self_outputs = self.self( 2025-12-04T09:35:37.8791294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8791481Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8792243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8792541Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.8793155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.8793292Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.8793307Z 2025-12-04T09:35:37.8793493Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8794281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8794411Z layer_outputs = layer_module( 2025-12-04T09:35:37.8794896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8795029Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8795659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8795785Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8796410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8796528Z self_outputs = self.self( 2025-12-04T09:35:37.8797146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8797332Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8798097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8798469Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8798484Z 2025-12-04T09:35:37.8798668Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8799463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8799593Z layer_outputs = layer_module( 2025-12-04T09:35:37.8800069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8800209Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8801338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8801477Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8802233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8802354Z self_outputs = self.self( 2025-12-04T09:35:37.8802980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8803168Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8803929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8804300Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8804321Z 2025-12-04T09:35:37.8804509Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8805315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8805439Z layer_outputs = layer_module( 2025-12-04T09:35:37.8805919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8806067Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8806693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8806822Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8807456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8807580Z self_outputs = self.self( 2025-12-04T09:35:37.8808202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8808378Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8809140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8809512Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8809527Z 2025-12-04T09:35:37.8809662Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8809807Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8809994Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8810782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8810914Z layer_outputs = layer_module( 2025-12-04T09:35:37.8811392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8811530Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8812166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8812293Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8813039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8813155Z self_outputs = self.self( 2025-12-04T09:35:37.8813757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.8813882Z attn_scores += diagonal_mask 2025-12-04T09:35:37.8813970Z 2025-12-04T09:35:37.8814154Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8815353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8815479Z layer_outputs = layer_module( 2025-12-04T09:35:37.8815961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8816106Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8816731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8816860Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8817491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8817613Z self_outputs = self.self( 2025-12-04T09:35:37.8818238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.8818370Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.8818385Z 2025-12-04T09:35:37.8818523Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8818718Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8819581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8819710Z layer_outputs = layer_module( 2025-12-04T09:35:37.8820189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8820320Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8820958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8821090Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8821723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8821840Z self_outputs = self.self( 2025-12-04T09:35:37.8822461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8822680Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8823480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8823813Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.8824224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8824396Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8824411Z 2025-12-04T09:35:37.8824601Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8825404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8825524Z layer_outputs = layer_module( 2025-12-04T09:35:37.8826012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8826144Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8826776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8826905Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8827688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8827888Z self_outputs = self.self( 2025-12-04T09:35:37.8828565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8828786Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8829577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8829825Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.8830540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.8830696Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.8831113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8831289Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8831305Z 2025-12-04T09:35:37.8831488Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8832292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8832413Z layer_outputs = layer_module( 2025-12-04T09:35:37.8832889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8833031Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8833651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8833786Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8834413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8834532Z self_outputs = self.self( 2025-12-04T09:35:37.8835166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8835378Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8836182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8836470Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8836486Z 2025-12-04T09:35:37.8836668Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8837462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8837586Z layer_outputs = layer_module( 2025-12-04T09:35:37.8838069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8838205Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8838835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8839078Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8839683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8839795Z self_outputs = self.self( 2025-12-04T09:35:37.8840405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8840665Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8841445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8841784Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8841800Z 2025-12-04T09:35:37.8841979Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8842747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8842866Z layer_outputs = layer_module( 2025-12-04T09:35:37.8843333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8843464Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8844071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8844212Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8844819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8844939Z self_outputs = self.self( 2025-12-04T09:35:37.8845543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.8845889Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.8845904Z 2025-12-04T09:35:37.8846045Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8846176Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8846352Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8847129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8847250Z layer_outputs = layer_module( 2025-12-04T09:35:37.8847726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8847857Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8848466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.8848614Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.8849162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.8849291Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.8849908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.8850097Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.8850708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.8850905Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.8851347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.8851469Z return self.act(input) 2025-12-04T09:35:37.8851484Z 2025-12-04T09:35:37.8851617Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8851753Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8851935Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8852707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8852888Z layer_outputs = layer_module( 2025-12-04T09:35:37.8853350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8853486Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8854166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8854294Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8854904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8855020Z self_outputs = self.self( 2025-12-04T09:35:37.8855619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8855799Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8856548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8856908Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8856928Z 2025-12-04T09:35:37.8857057Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8857232Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8858008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8858124Z layer_outputs = layer_module( 2025-12-04T09:35:37.8858599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8858727Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8859334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8859473Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8860086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8860207Z self_outputs = self.self( 2025-12-04T09:35:37.8860807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8860976Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8861724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 789, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8862001Z key = self._chunk(key, window_overlap, getattr(self.config, "onnx_export", False)) 2025-12-04T09:35:37.8862596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 718, in _chunk 2025-12-04T09:35:37.8862731Z hidden_states = hidden_states.view( 2025-12-04T09:35:37.8862746Z 2025-12-04T09:35:37.8862924Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8863692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8863813Z layer_outputs = layer_module( 2025-12-04T09:35:37.8864276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8864413Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8865013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8865141Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8865802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8865913Z self_outputs = self.self( 2025-12-04T09:35:37.8866576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8866747Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8867778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8868143Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8868158Z 2025-12-04T09:35:37.8868343Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8869138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8869265Z layer_outputs = layer_module( 2025-12-04T09:35:37.8869750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8869891Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8870516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8870652Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8871277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8871394Z self_outputs = self.self( 2025-12-04T09:35:37.8872023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8872199Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8872970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8873337Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8873352Z 2025-12-04T09:35:37.8873534Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8874330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8874448Z layer_outputs = layer_module( 2025-12-04T09:35:37.8874935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8875067Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8875690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8875831Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8876458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8876584Z self_outputs = self.self( 2025-12-04T09:35:37.8877272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 523, in forward 2025-12-04T09:35:37.8877559Z attn_scores = self._sliding_chunks_query_key_matmul( 2025-12-04T09:35:37.8878334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 795, in _sliding_chunks_query_key_matmul 2025-12-04T09:35:37.8878695Z diagonal_chunked_attention_scores = torch.einsum("bcxd,bcyd->bcxy", (query, key)) # multiply 2025-12-04T09:35:37.8878711Z 2025-12-04T09:35:37.8878929Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8879065Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8879252Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8880123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8880248Z layer_outputs = layer_module( 2025-12-04T09:35:37.8880730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8880874Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8881503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8881643Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8882334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8882460Z self_outputs = self.self( 2025-12-04T09:35:37.8883087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 540, in forward 2025-12-04T09:35:37.8883212Z attn_scores += diagonal_mask 2025-12-04T09:35:37.8883227Z 2025-12-04T09:35:37.8883426Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8884220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8884339Z layer_outputs = layer_module( 2025-12-04T09:35:37.8884822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8884955Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8885587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8885730Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8886361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8886483Z self_outputs = self.self( 2025-12-04T09:35:37.8887105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 578, in forward 2025-12-04T09:35:37.8887232Z attn_probs = nn.functional.softmax( 2025-12-04T09:35:37.8887248Z 2025-12-04T09:35:37.8887388Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8887573Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8888376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8888501Z layer_outputs = layer_module( 2025-12-04T09:35:37.8889090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8889236Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8889844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8889969Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8890582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8890693Z self_outputs = self.self( 2025-12-04T09:35:37.8891307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8891511Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8892277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 862, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8892671Z padded_value = nn.functional.pad(value, (0, 0, window_overlap, window_overlap), value=-1) 2025-12-04T09:35:37.8893113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8893283Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8893297Z 2025-12-04T09:35:37.8893475Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8894240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8894361Z layer_outputs = layer_module( 2025-12-04T09:35:37.8894827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8894965Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8895573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8895696Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8896315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8896433Z self_outputs = self.self( 2025-12-04T09:35:37.8897039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8897260Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8898031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 875, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8898286Z chunked_attn_probs = self._pad_and_diagonalize(chunked_attn_probs) 2025-12-04T09:35:37.8898974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 698, in _pad_and_diagonalize 2025-12-04T09:35:37.8899133Z chunked_hidden_states = nn.functional.pad( 2025-12-04T09:35:37.8899539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/functional.py", line 5418, in pad 2025-12-04T09:35:37.8899708Z return torch._C._nn.pad(input, pad, mode, value) 2025-12-04T09:35:37.8899723Z 2025-12-04T09:35:37.8899915Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8900682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8901315Z layer_outputs = layer_module( 2025-12-04T09:35:37.8901908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8902054Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8902703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8902837Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8903466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8903599Z self_outputs = self.self( 2025-12-04T09:35:37.8904222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8904437Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8905239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8905656Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8905673Z 2025-12-04T09:35:37.8905877Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8906739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8906863Z layer_outputs = layer_module( 2025-12-04T09:35:37.8907471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8907608Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8908255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8908386Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8909010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8909153Z self_outputs = self.self( 2025-12-04T09:35:37.8909782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 612, in forward 2025-12-04T09:35:37.8910009Z attn_output = self._sliding_chunks_matmul_attn_probs_value( 2025-12-04T09:35:37.8910806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 877, in _sliding_chunks_matmul_attn_probs_value 2025-12-04T09:35:37.8911100Z context = torch.einsum("bcwd,bcdh->bcwh", (chunked_attn_probs, chunked_value)) 2025-12-04T09:35:37.8911115Z 2025-12-04T09:35:37.8911316Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8912116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8912256Z layer_outputs = layer_module( 2025-12-04T09:35:37.8912737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8912870Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8913517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1198, in forward 2025-12-04T09:35:37.8913646Z self_attn_outputs = self.attention( 2025-12-04T09:35:37.8914271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1134, in forward 2025-12-04T09:35:37.8914397Z self_outputs = self.self( 2025-12-04T09:35:37.8915017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 617, in forward 2025-12-04T09:35:37.8915381Z attn_output = attn_output.transpose(0, 1).reshape(seq_len, batch_size, embed_dim).contiguous() 2025-12-04T09:35:37.8915401Z 2025-12-04T09:35:37.8915539Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8915672Z cudagraph partition due to non gpu ops 2025-12-04T09:35:37.8915860Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:35:37.8916654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1258, in torch_dynamo_resume_in_forward_at_1243 2025-12-04T09:35:37.8916780Z layer_outputs = layer_module( 2025-12-04T09:35:37.8917261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:35:37.8917392Z return super().__call__(*args, **kwargs) 2025-12-04T09:35:37.8918024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1210, in forward 2025-12-04T09:35:37.8918167Z layer_output = apply_chunking_to_forward( 2025-12-04T09:35:37.8918907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:35:37.8919045Z return forward_fn(*input_tensors) 2025-12-04T09:35:37.8919716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1217, in ff_chunk 2025-12-04T09:35:37.8919923Z intermediate_output = self.intermediate(attn_output) 2025-12-04T09:35:37.8920531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1160, in forward 2025-12-04T09:35:37.8920721Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:35:37.8921169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:35:37.8921281Z return self.act(input) 2025-12-04T09:35:37.8921296Z 2025-12-04T09:35:37.8921433Z cudagraph partition due to non gpu ops 2025-12-04T09:36:44.7405372Z cudagraph partition due to non gpu ops 2025-12-04T09:36:44.7407366Z cudagraph partition due to non gpu ops 2025-12-04T09:36:44.7407896Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:36:44.7409103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/longformer/modeling_longformer.py", line 1722, in torch_dynamo_resume_in_forward_at_1702 2025-12-04T09:36:44.7410426Z masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:36:44.7410946Z 2025-12-04T09:36:46.8467307Z Compilation time (from dynamo_timed): 113.011428491 2025-12-04T09:36:46.8839719Z pass 2025-12-04T09:36:46.8840330Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:36:46.8842049Z TIMING: gc:0.0048 entire_frame_compile:113.01143 _recursive_pre_grad_passes:0.17787 _recursive_joint_graph_passes:1.6195 _recursive_post_grad_passes:1.50423 async_compile.wait:4.22064 code_gen:73.28748 inductor_compile:82.55573 backend_compile:104.47273 total_wall_time:113.01143 2025-12-04T09:36:46.8844249Z STATS: call_* op count: 1787 | FakeTensorMode.__torch_dispatch__:56269 | FakeTensor.__torch_dispatch__:8931 | ProxyTorchDispatchMode.__torch_dispatch__:10841 2025-12-04T09:36:46.8845311Z Dynamo produced 4 graphs covering 1787 ops with 4 graph breaks (1 unique) 2025-12-04T09:36:50.2348781Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:36:50.2350417Z import pynvml # type: ignore[import] 2025-12-04T09:36:55.2092151Z 2025-12-04T09:37:00.0564509Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:37:00.0564931Z loading model: 0it [00:04, ?it/s] 2025-12-04T09:37:00.0591965Z cpu eval BartForCausalLM 2025-12-04T09:37:02.4688751Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:37:03.5064241Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:37:04.6459965Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:37:19.5194804Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5195299Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5195701Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5196083Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5196501Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5196897Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5197294Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5197676Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5198068Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5198464Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5199295Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5199690Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5200097Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5200477Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5201616Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5202017Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5202414Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5202793Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5203195Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5203647Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5204391Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5205074Z res = mod(**inputs) 2025-12-04T09:37:19.5205845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5206680Z outputs = self.model.decoder( 2025-12-04T09:37:19.5207481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5208311Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5209042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5209797Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5210600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5211411Z return func(*args, **kwargs) 2025-12-04T09:37:19.5212195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5213057Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5213898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5216239Z return func(*args, **kwargs) 2025-12-04T09:37:19.5217021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5217892Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5218846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5219887Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5220273Z 2025-12-04T09:37:19.5220418Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5220816Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5221308Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5222043Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5222719Z res = mod(**inputs) 2025-12-04T09:37:19.5223452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5224285Z outputs = self.model.decoder( 2025-12-04T09:37:19.5225092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5225919Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5226641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5227531Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5228341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5229148Z return func(*args, **kwargs) 2025-12-04T09:37:19.5229926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5230980Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5231876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5232599Z return self.act(input) 2025-12-04T09:37:19.5232823Z 2025-12-04T09:37:19.5232967Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5233374Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5233781Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5234162Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5234551Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5234948Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5235326Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5235718Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5236169Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5236898Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5237570Z res = mod(**inputs) 2025-12-04T09:37:19.5238323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5239159Z outputs = self.model.decoder( 2025-12-04T09:37:19.5239946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5240769Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5241496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5242250Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5243053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5243864Z return func(*args, **kwargs) 2025-12-04T09:37:19.5244638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5245497Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5246352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5247139Z return func(*args, **kwargs) 2025-12-04T09:37:19.5247936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5248792Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5249749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5250791Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5251175Z 2025-12-04T09:37:19.5251339Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5251726Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5252180Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5252921Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5253583Z res = mod(**inputs) 2025-12-04T09:37:19.5254335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5255165Z outputs = self.model.decoder( 2025-12-04T09:37:19.5255960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5256763Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5257489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5258257Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5259133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5259918Z return func(*args, **kwargs) 2025-12-04T09:37:19.5260762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5261677Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5262480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5263214Z return self.act(input) 2025-12-04T09:37:19.5263439Z 2025-12-04T09:37:19.5263583Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5263982Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5264358Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5264749Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5265141Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5265518Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5265905Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5266290Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5266811Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5267555Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5268220Z res = mod(**inputs) 2025-12-04T09:37:19.5268968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5269780Z outputs = self.model.decoder( 2025-12-04T09:37:19.5270573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5271399Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5272109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5272880Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5273677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5274480Z return func(*args, **kwargs) 2025-12-04T09:37:19.5275241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5276107Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5277112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5277938Z return func(*args, **kwargs) 2025-12-04T09:37:19.5278723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5279589Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5280553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5281629Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5282028Z 2025-12-04T09:37:19.5282176Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5282578Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5283023Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5283746Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5284415Z res = mod(**inputs) 2025-12-04T09:37:19.5285162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5285981Z outputs = self.model.decoder( 2025-12-04T09:37:19.5286769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5288161Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5288893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5289710Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5290516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5291308Z return func(*args, **kwargs) 2025-12-04T09:37:19.5292068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5292975Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5293786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5294624Z return self.act(input) 2025-12-04T09:37:19.5294830Z 2025-12-04T09:37:19.5294969Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5295354Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5295734Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5296097Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5296481Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5296857Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5297234Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5297600Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5298026Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5298738Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5299372Z res = mod(**inputs) 2025-12-04T09:37:19.5300097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5301635Z outputs = self.model.decoder( 2025-12-04T09:37:19.5302427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5303249Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5303983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5304750Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5305539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5306334Z return func(*args, **kwargs) 2025-12-04T09:37:19.5307222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5308099Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5308925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5309724Z return func(*args, **kwargs) 2025-12-04T09:37:19.5310493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5311347Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5312304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5313332Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5313711Z 2025-12-04T09:37:19.5313866Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5314251Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5314697Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5315437Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5316095Z res = mod(**inputs) 2025-12-04T09:37:19.5316974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5317809Z outputs = self.model.decoder( 2025-12-04T09:37:19.5318793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5319575Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5320280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5321029Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5321811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5322569Z return func(*args, **kwargs) 2025-12-04T09:37:19.5323327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5324218Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5324991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5325695Z return self.act(input) 2025-12-04T09:37:19.5325914Z 2025-12-04T09:37:19.5326053Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5326441Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5326806Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5327188Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5327569Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5327932Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5328310Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5328687Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5329102Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5329819Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5330472Z res = mod(**inputs) 2025-12-04T09:37:19.5331192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5331981Z outputs = self.model.decoder( 2025-12-04T09:37:19.5332745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5333547Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5334238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5334978Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5335749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5336519Z return func(*args, **kwargs) 2025-12-04T09:37:19.5337262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5338107Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5338932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5340085Z return func(*args, **kwargs) 2025-12-04T09:37:19.5340864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5341727Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5342680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5343692Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5344087Z 2025-12-04T09:37:19.5344231Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5344745Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5345314Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5346044Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5346809Z res = mod(**inputs) 2025-12-04T09:37:19.5347643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5348469Z outputs = self.model.decoder( 2025-12-04T09:37:19.5349268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5350087Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5350816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5351569Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5352377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5353177Z return func(*args, **kwargs) 2025-12-04T09:37:19.5353941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5354847Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5355659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5356384Z return self.act(input) 2025-12-04T09:37:19.5356589Z 2025-12-04T09:37:19.5356731Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5357129Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5357520Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5357895Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5358283Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5358668Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5359164Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5359533Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5359967Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5360687Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5361325Z res = mod(**inputs) 2025-12-04T09:37:19.5362046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5362849Z outputs = self.model.decoder( 2025-12-04T09:37:19.5363610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5364393Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5375076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5375976Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5376782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5377570Z return func(*args, **kwargs) 2025-12-04T09:37:19.5378340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5379174Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5379998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5380776Z return func(*args, **kwargs) 2025-12-04T09:37:19.5381518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5382358Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5383261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5384363Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5384731Z 2025-12-04T09:37:19.5384874Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5385328Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5385770Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5386480Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5387278Z res = mod(**inputs) 2025-12-04T09:37:19.5388201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5389033Z outputs = self.model.decoder( 2025-12-04T09:37:19.5389813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5390642Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5391370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5392142Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5392935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5393736Z return func(*args, **kwargs) 2025-12-04T09:37:19.5394516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5395412Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5396227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5396953Z return self.act(input) 2025-12-04T09:37:19.5397158Z 2025-12-04T09:37:19.5397315Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5397704Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5398099Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5398493Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5398872Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5399267Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5399657Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5400034Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5400482Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5406647Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5407329Z res = mod(**inputs) 2025-12-04T09:37:19.5408067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5408894Z outputs = self.model.decoder( 2025-12-04T09:37:19.5409690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5410514Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5411241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5412016Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5412820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5413601Z return func(*args, **kwargs) 2025-12-04T09:37:19.5414481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5415324Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5416131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5416906Z return func(*args, **kwargs) 2025-12-04T09:37:19.5417809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5418650Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5420881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5421908Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5422291Z 2025-12-04T09:37:19.5422432Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5422822Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5423243Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5423957Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5424610Z res = mod(**inputs) 2025-12-04T09:37:19.5425326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5426142Z outputs = self.model.decoder( 2025-12-04T09:37:19.5427021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5428023Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5428744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5429516Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5430320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5431112Z return func(*args, **kwargs) 2025-12-04T09:37:19.5431889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5432796Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5433627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5434336Z return self.act(input) 2025-12-04T09:37:19.5434557Z 2025-12-04T09:37:19.5434701Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5435107Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5435490Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5435885Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5436280Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5436674Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5437048Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5437438Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5437886Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5438610Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5439385Z res = mod(**inputs) 2025-12-04T09:37:19.5440118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5440925Z outputs = self.model.decoder( 2025-12-04T09:37:19.5441692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5442492Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5443195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5443930Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5444707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5445475Z return func(*args, **kwargs) 2025-12-04T09:37:19.5446233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5447141Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5447963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5448810Z return func(*args, **kwargs) 2025-12-04T09:37:19.5449549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5450385Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5451310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5452310Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5452675Z 2025-12-04T09:37:19.5452813Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5453202Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5453641Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5454342Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5454988Z res = mod(**inputs) 2025-12-04T09:37:19.5455715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5456523Z outputs = self.model.decoder( 2025-12-04T09:37:19.5457278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5458074Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5458778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5459529Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5460292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5461071Z return func(*args, **kwargs) 2025-12-04T09:37:19.5461825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5462697Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5463486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5469268Z return self.act(input) 2025-12-04T09:37:19.5469481Z 2025-12-04T09:37:19.5469639Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5470028Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5470426Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5470820Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5471199Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5471588Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5471981Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5472366Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5472814Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5473557Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5474237Z res = mod(**inputs) 2025-12-04T09:37:19.5474969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5475801Z outputs = self.model.decoder( 2025-12-04T09:37:19.5476600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5477411Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5478138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5479024Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5479905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5480669Z return func(*args, **kwargs) 2025-12-04T09:37:19.5481500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5482340Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5483146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5483918Z return func(*args, **kwargs) 2025-12-04T09:37:19.5484677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5485517Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5486426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5487430Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5487811Z 2025-12-04T09:37:19.5487950Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5488337Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5488764Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5489479Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5490131Z res = mod(**inputs) 2025-12-04T09:37:19.5490846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5491652Z outputs = self.model.decoder( 2025-12-04T09:37:19.5492419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5493214Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5493903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5494652Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5495440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5496194Z return func(*args, **kwargs) 2025-12-04T09:37:19.5496941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5497825Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5498608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5499296Z return self.act(input) 2025-12-04T09:37:19.5499511Z 2025-12-04T09:37:19.5499648Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5500034Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5500418Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5501334Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5501786Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5502182Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5502565Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5502958Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5503404Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5504134Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5504800Z res = mod(**inputs) 2025-12-04T09:37:19.5505545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5506373Z outputs = self.model.decoder( 2025-12-04T09:37:19.5507267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5508233Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5508962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5509716Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5510592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5511373Z return func(*args, **kwargs) 2025-12-04T09:37:19.5512133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5512982Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5513809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5514600Z return func(*args, **kwargs) 2025-12-04T09:37:19.5515363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5516213Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5517156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5518162Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5518647Z 2025-12-04T09:37:19.5518789Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5519155Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5519576Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5520277Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5520904Z res = mod(**inputs) 2025-12-04T09:37:19.5521615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5522406Z outputs = self.model.decoder( 2025-12-04T09:37:19.5523152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5523931Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5524626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5525377Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5526133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5527205Z return func(*args, **kwargs) 2025-12-04T09:37:19.5528027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5528915Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5529698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5530409Z return self.act(input) 2025-12-04T09:37:19.5530612Z 2025-12-04T09:37:19.5530757Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5531134Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5531513Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5531888Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5532318Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5532689Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5533064Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5533433Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5533856Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5534580Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5535226Z res = mod(**inputs) 2025-12-04T09:37:19.5535946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5536848Z outputs = self.model.decoder( 2025-12-04T09:37:19.5537624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5538488Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5539300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5540026Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5540786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5541527Z return func(*args, **kwargs) 2025-12-04T09:37:19.5542262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5543088Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5543891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5544640Z return func(*args, **kwargs) 2025-12-04T09:37:19.5545383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5546203Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5547385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5548395Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5548779Z 2025-12-04T09:37:19.5548921Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5549304Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5549725Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5550455Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5551110Z res = mod(**inputs) 2025-12-04T09:37:19.5551847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5552661Z outputs = self.model.decoder( 2025-12-04T09:37:19.5553443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5554246Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5554947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5555691Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5556483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5557266Z return func(*args, **kwargs) 2025-12-04T09:37:19.5558022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5559017Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5559791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5560467Z return self.act(input) 2025-12-04T09:37:19.5560673Z 2025-12-04T09:37:19.5560806Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5561174Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5561539Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5561900Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5562264Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5562626Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5562981Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5563343Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5563828Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5564520Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5565149Z res = mod(**inputs) 2025-12-04T09:37:19.5565931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5566726Z outputs = self.model.decoder( 2025-12-04T09:37:19.5567467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5568242Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5568927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5569644Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5570409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5571171Z return func(*args, **kwargs) 2025-12-04T09:37:19.5571912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 418, in forward 2025-12-04T09:37:19.5572732Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:37:19.5573536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5574294Z return func(*args, **kwargs) 2025-12-04T09:37:19.5575027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 258, in forward 2025-12-04T09:37:19.5575839Z attn_output, attn_weights = attention_interface( 2025-12-04T09:37:19.5576748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:37:19.5577726Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:19.5578091Z 2025-12-04T09:37:19.5578228Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5578598Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5579011Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5579713Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5580338Z res = mod(**inputs) 2025-12-04T09:37:19.5581045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1902, in forward 2025-12-04T09:37:19.5581833Z outputs = self.model.decoder( 2025-12-04T09:37:19.5582581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1122, in forward 2025-12-04T09:37:19.5583365Z layer_outputs = decoder_layer( 2025-12-04T09:37:19.5584053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:19.5584782Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:19.5585532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:19.5586299Z return func(*args, **kwargs) 2025-12-04T09:37:19.5587296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 450, in forward 2025-12-04T09:37:19.5588189Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:37:19.5589111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:19.5589897Z return self.act(input) 2025-12-04T09:37:19.5590100Z 2025-12-04T09:37:19.5590246Z cudagraph partition due to non gpu ops 2025-12-04T09:37:19.5590677Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5591401Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5592133Z res = mod(**inputs) 2025-12-04T09:37:19.5592874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1918, in forward 2025-12-04T09:37:19.5593682Z logits = self.lm_head(outputs[0]) 2025-12-04T09:37:19.5593994Z 2025-12-04T09:37:19.5594211Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:19.5594956Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:19.5595601Z res = mod(**inputs) 2025-12-04T09:37:19.5596333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bart/modeling_bart.py", line 1924, in forward 2025-12-04T09:37:19.5597285Z loss = loss_fct(logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:37:19.5597686Z 2025-12-04T09:37:26.9285691Z Compilation time (from dynamo_timed): 20.431428249 2025-12-04T09:37:26.9607215Z pass 2025-12-04T09:37:26.9607883Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:37:26.9609604Z TIMING: _recursive_pre_grad_passes:0.04791 _recursive_joint_graph_passes:0.88533 _recursive_post_grad_passes:0.08818 async_compile.wait:1.14966 code_gen:6.34666 inductor_compile:10.36982 backend_compile:16.87129 gc:0.0004 entire_frame_compile:20.43143 total_wall_time:20.43143 2025-12-04T09:37:26.9611573Z STATS: call_* op count: 336 | FakeTensorMode.__torch_dispatch__:18857 | FakeTensor.__torch_dispatch__:3477 | ProxyTorchDispatchMode.__torch_dispatch__:2854 2025-12-04T09:37:26.9612633Z Dynamo produced 1 graphs covering 336 ops with 0 graph breaks (0 unique) 2025-12-04T09:37:29.8600612Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:37:29.8603015Z import pynvml # type: ignore[import] 2025-12-04T09:37:34.7955154Z 2025-12-04T09:37:36.9006326Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:37:36.9006768Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:37:36.9024853Z cpu eval BertForMaskedLM 2025-12-04T09:37:37.6671979Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:37:38.0180804Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:37:38.3649268Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:37:53.6897807Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6898275Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6898684Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6899081Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6899455Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6899940Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6900326Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6900695Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6901806Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6902233Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6902630Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6903050Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6903449Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6903845Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6904224Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6904647Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6905045Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6905445Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6905830Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6906294Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.6907544Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.6908218Z res = mod(**inputs) 2025-12-04T09:37:53.6909112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.6910131Z outputs = self.bert( 2025-12-04T09:37:53.6910885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.6911723Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.6912534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.6913424Z layer_outputs = layer_module( 2025-12-04T09:37:53.6914136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.6914922Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.6915740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.6916552Z return func(*args, **kwargs) 2025-12-04T09:37:53.6917328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.6918181Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.6919009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.6919803Z return func(*args, **kwargs) 2025-12-04T09:37:53.6920594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.6921415Z self_outputs = self.self( 2025-12-04T09:37:53.6922173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.6922961Z return func(*args, **kwargs) 2025-12-04T09:37:53.6923743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.6924689Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.6925077Z 2025-12-04T09:37:53.6925240Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6925632Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6926085Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.6926924Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.6927569Z res = mod(**inputs) 2025-12-04T09:37:53.6928306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.6929113Z outputs = self.bert( 2025-12-04T09:37:53.6929843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.6930655Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.6931439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.6932243Z layer_outputs = layer_module( 2025-12-04T09:37:53.6932932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.6933682Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.6934463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.6935244Z return func(*args, **kwargs) 2025-12-04T09:37:53.6935991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.6936918Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.6937761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.6938576Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.6939496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.6940453Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.6941341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.6942195Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.6942977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.6943691Z return self.act(input) 2025-12-04T09:37:53.6943895Z 2025-12-04T09:37:53.6944062Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6944574Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6944963Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6945351Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6945726Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6946113Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6946503Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6947014Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6947639Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.6948387Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.6949070Z res = mod(**inputs) 2025-12-04T09:37:53.6949810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.6950626Z outputs = self.bert( 2025-12-04T09:37:53.6951395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.6952219Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.6953022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.6953847Z layer_outputs = layer_module( 2025-12-04T09:37:53.6954575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.6955330Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.6956134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.6956943Z return func(*args, **kwargs) 2025-12-04T09:37:53.6957714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.6958566Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.6959490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.6960262Z return func(*args, **kwargs) 2025-12-04T09:37:53.6961010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.6961802Z self_outputs = self.self( 2025-12-04T09:37:53.6962537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.6963307Z return func(*args, **kwargs) 2025-12-04T09:37:53.6964050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.6964955Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.6965325Z 2025-12-04T09:37:53.6965477Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6965942Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6966385Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.6967109Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.6967831Z res = mod(**inputs) 2025-12-04T09:37:53.6968550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.6969343Z outputs = self.bert( 2025-12-04T09:37:53.6970082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.6970871Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.6971645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.6978643Z layer_outputs = layer_module( 2025-12-04T09:37:53.6979381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.6980143Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.6980961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.6981761Z return func(*args, **kwargs) 2025-12-04T09:37:53.6982522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.6983375Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.6984234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.6985188Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.6986016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.6987103Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.6988205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.6989105Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.6989889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.6990623Z return self.act(input) 2025-12-04T09:37:53.6990834Z 2025-12-04T09:37:53.6990994Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6991381Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6991771Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6992161Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6992556Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6992932Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6993319Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6993715Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.6994146Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.6994886Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.6995563Z res = mod(**inputs) 2025-12-04T09:37:53.6996300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.6997121Z outputs = self.bert( 2025-12-04T09:37:53.6997882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.6998822Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.6999584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7000380Z layer_outputs = layer_module( 2025-12-04T09:37:53.7001850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7002610Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7003592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7004404Z return func(*args, **kwargs) 2025-12-04T09:37:53.7005184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.7006014Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.7006827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7007620Z return func(*args, **kwargs) 2025-12-04T09:37:53.7008399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.7009202Z self_outputs = self.self( 2025-12-04T09:37:53.7009957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7010757Z return func(*args, **kwargs) 2025-12-04T09:37:53.7011521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.7012452Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.7012848Z 2025-12-04T09:37:53.7013104Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7013496Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7013919Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7014636Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7015285Z res = mod(**inputs) 2025-12-04T09:37:53.7015995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7016797Z outputs = self.bert( 2025-12-04T09:37:53.7017532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7018336Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7019095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7019886Z layer_outputs = layer_module( 2025-12-04T09:37:53.7020587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7021317Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7022093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7022871Z return func(*args, **kwargs) 2025-12-04T09:37:53.7023630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.7024435Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.7025271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.7026092Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.7027019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.7028163Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.7029069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.7029962Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.7030742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.7031587Z return self.act(input) 2025-12-04T09:37:53.7031812Z 2025-12-04T09:37:53.7031957Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7032359Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7032807Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7033205Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7033597Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7033977Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7034405Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7034840Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7039123Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7039870Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7040546Z res = mod(**inputs) 2025-12-04T09:37:53.7041300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7042113Z outputs = self.bert( 2025-12-04T09:37:53.7042871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7043708Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7044497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7045324Z layer_outputs = layer_module( 2025-12-04T09:37:53.7046044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7046918Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7047682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7048462Z return func(*args, **kwargs) 2025-12-04T09:37:53.7049228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.7050042Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.7050818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7051598Z return func(*args, **kwargs) 2025-12-04T09:37:53.7052357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.7053130Z self_outputs = self.self( 2025-12-04T09:37:53.7053871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7054647Z return func(*args, **kwargs) 2025-12-04T09:37:53.7055407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.7056311Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.7056695Z 2025-12-04T09:37:53.7056838Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7057236Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7057668Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7058389Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7059051Z res = mod(**inputs) 2025-12-04T09:37:53.7059787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7060574Z outputs = self.bert( 2025-12-04T09:37:53.7061315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7062124Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7062878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7063771Z layer_outputs = layer_module( 2025-12-04T09:37:53.7064471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7065284Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7066044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7066916Z return func(*args, **kwargs) 2025-12-04T09:37:53.7067849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.7068672Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.7069522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.7070371Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.7071239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.7072210Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.7073129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.7074031Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.7074825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.7075537Z return self.act(input) 2025-12-04T09:37:53.7075758Z 2025-12-04T09:37:53.7075901Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7076298Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7076678Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7077067Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7077460Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7077844Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7078236Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7078629Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7079082Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7079905Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7080553Z res = mod(**inputs) 2025-12-04T09:37:53.7081279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7082055Z outputs = self.bert( 2025-12-04T09:37:53.7082784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7083589Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7084360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7085142Z layer_outputs = layer_module( 2025-12-04T09:37:53.7085845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7086590Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7087360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7088132Z return func(*args, **kwargs) 2025-12-04T09:37:53.7088883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.7089697Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.7090471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7091238Z return func(*args, **kwargs) 2025-12-04T09:37:53.7092069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.7092859Z self_outputs = self.self( 2025-12-04T09:37:53.7093642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7094422Z return func(*args, **kwargs) 2025-12-04T09:37:53.7095175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.7096065Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.7096445Z 2025-12-04T09:37:53.7096582Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7097035Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7097771Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7098493Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7099172Z res = mod(**inputs) 2025-12-04T09:37:53.7099919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7100723Z outputs = self.bert( 2025-12-04T09:37:53.7101886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7102889Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7103688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7104492Z layer_outputs = layer_module( 2025-12-04T09:37:53.7105220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7105992Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7106885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7107698Z return func(*args, **kwargs) 2025-12-04T09:37:53.7108479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.7109328Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.7110170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.7111016Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.7111884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.7112862Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.7113762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.7114659Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.7115463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.7116177Z return self.act(input) 2025-12-04T09:37:53.7116402Z 2025-12-04T09:37:53.7116549Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7116951Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7117354Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7117738Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7118137Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7118641Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7119008Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7119393Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7119835Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7120588Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7121412Z res = mod(**inputs) 2025-12-04T09:37:53.7122145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7122937Z outputs = self.bert( 2025-12-04T09:37:53.7123741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7124553Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7125331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7126105Z layer_outputs = layer_module( 2025-12-04T09:37:53.7126810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7127559Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7128343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7129110Z return func(*args, **kwargs) 2025-12-04T09:37:53.7129869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.7130686Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.7131460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7132231Z return func(*args, **kwargs) 2025-12-04T09:37:53.7132984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.7133773Z self_outputs = self.self( 2025-12-04T09:37:53.7134492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7135263Z return func(*args, **kwargs) 2025-12-04T09:37:53.7136013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.7136915Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.7137281Z 2025-12-04T09:37:53.7137420Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7137814Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7138254Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7138963Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7139613Z res = mod(**inputs) 2025-12-04T09:37:53.7140342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7141135Z outputs = self.bert( 2025-12-04T09:37:53.7141850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7142661Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7143434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7144207Z layer_outputs = layer_module( 2025-12-04T09:37:53.7144910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7145655Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7146431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7147448Z return func(*args, **kwargs) 2025-12-04T09:37:53.7148238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.7149086Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.7149929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.7150858Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.7151728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.7152771Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.7153667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.7154560Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.7155353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.7166767Z return self.act(input) 2025-12-04T09:37:53.7167015Z 2025-12-04T09:37:53.7167184Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7167579Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7167999Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7168398Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7168775Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7169165Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7169552Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7169945Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7170395Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7171147Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7171940Z res = mod(**inputs) 2025-12-04T09:37:53.7172669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7173472Z outputs = self.bert( 2025-12-04T09:37:53.7174212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7175012Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7175793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7176591Z layer_outputs = layer_module( 2025-12-04T09:37:53.7177304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7178042Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7178830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7179605Z return func(*args, **kwargs) 2025-12-04T09:37:53.7180364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.7181170Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.7181959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7182742Z return func(*args, **kwargs) 2025-12-04T09:37:53.7183489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.7184294Z self_outputs = self.self( 2025-12-04T09:37:53.7185034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7185810Z return func(*args, **kwargs) 2025-12-04T09:37:53.7186559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.7187890Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.7188273Z 2025-12-04T09:37:53.7188437Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7188830Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7189293Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7190165Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7190843Z res = mod(**inputs) 2025-12-04T09:37:53.7191652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7192472Z outputs = self.bert( 2025-12-04T09:37:53.7193233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7194048Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7194847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7195677Z layer_outputs = layer_module( 2025-12-04T09:37:53.7196407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7197172Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7197982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7198789Z return func(*args, **kwargs) 2025-12-04T09:37:53.7199657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.7200477Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.7201908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.7202766Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.7203620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.7204597Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.7205505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.7206410Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.7207203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.7207928Z return self.act(input) 2025-12-04T09:37:53.7208138Z 2025-12-04T09:37:53.7208296Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7208688Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7209084Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7209476Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7209877Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7210256Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7210650Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7211049Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7211487Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7212234Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7212914Z res = mod(**inputs) 2025-12-04T09:37:53.7213767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7214566Z outputs = self.bert( 2025-12-04T09:37:53.7215301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7216107Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7216866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7217659Z layer_outputs = layer_module( 2025-12-04T09:37:53.7218363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7219238Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7220024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7220802Z return func(*args, **kwargs) 2025-12-04T09:37:53.7221642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.7228659Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.7229488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7230301Z return func(*args, **kwargs) 2025-12-04T09:37:53.7231085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.7231889Z self_outputs = self.self( 2025-12-04T09:37:53.7232653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7233471Z return func(*args, **kwargs) 2025-12-04T09:37:53.7234242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.7235184Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.7235703Z 2025-12-04T09:37:53.7235859Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7236334Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7236901Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7237741Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7238621Z res = mod(**inputs) 2025-12-04T09:37:53.7239343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7240177Z outputs = self.bert( 2025-12-04T09:37:53.7240972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7241783Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7242561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7243343Z layer_outputs = layer_module( 2025-12-04T09:37:53.7244042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7244790Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7245571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7246481Z return func(*args, **kwargs) 2025-12-04T09:37:53.7247317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.7248146Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.7248966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.7249795Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.7250644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.7251600Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.7252467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.7253339Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.7254117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.7254826Z return self.act(input) 2025-12-04T09:37:53.7255029Z 2025-12-04T09:37:53.7255295Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7255691Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7256082Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7256456Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7256898Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7257285Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7257653Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7258035Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7258471Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7259196Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7259834Z res = mod(**inputs) 2025-12-04T09:37:53.7260562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7261362Z outputs = self.bert( 2025-12-04T09:37:53.7262087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7262888Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7263663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7264452Z layer_outputs = layer_module( 2025-12-04T09:37:53.7265141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7265887Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7266666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7267711Z return func(*args, **kwargs) 2025-12-04T09:37:53.7268490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.7269338Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.7270151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7270938Z return func(*args, **kwargs) 2025-12-04T09:37:53.7271720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.7272538Z self_outputs = self.self( 2025-12-04T09:37:53.7273278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7274076Z return func(*args, **kwargs) 2025-12-04T09:37:53.7274862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.7275775Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.7276170Z 2025-12-04T09:37:53.7276311Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7276715Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7277149Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7277890Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7278564Z res = mod(**inputs) 2025-12-04T09:37:53.7279405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7280184Z outputs = self.bert( 2025-12-04T09:37:53.7280918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7281722Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7282483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7283273Z layer_outputs = layer_module( 2025-12-04T09:37:53.7284055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7287305Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7288213Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7289015Z return func(*args, **kwargs) 2025-12-04T09:37:53.7289789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.7290636Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.7291484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.7292325Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.7293171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.7294137Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.7295026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.7295914Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.7296693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.7297498Z return self.act(input) 2025-12-04T09:37:53.7297703Z 2025-12-04T09:37:53.7297840Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7298217Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7298578Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7298944Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7299311Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7299681Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7300039Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7300405Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7301229Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7302134Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7302802Z res = mod(**inputs) 2025-12-04T09:37:53.7303535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7304334Z outputs = self.bert( 2025-12-04T09:37:53.7305071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7305882Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7306667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7307547Z layer_outputs = layer_module( 2025-12-04T09:37:53.7308255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7308999Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7309787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7310564Z return func(*args, **kwargs) 2025-12-04T09:37:53.7311320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.7312141Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.7312923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7313698Z return func(*args, **kwargs) 2025-12-04T09:37:53.7314459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.7315390Z self_outputs = self.self( 2025-12-04T09:37:53.7316121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7316899Z return func(*args, **kwargs) 2025-12-04T09:37:53.7317742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.7318767Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.7319132Z 2025-12-04T09:37:53.7319268Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7319643Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7320063Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7320752Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7321387Z res = mod(**inputs) 2025-12-04T09:37:53.7322093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7322867Z outputs = self.bert( 2025-12-04T09:37:53.7323579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7324368Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7325159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7325923Z layer_outputs = layer_module( 2025-12-04T09:37:53.7326607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7327333Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7328098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7328850Z return func(*args, **kwargs) 2025-12-04T09:37:53.7329590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.7330394Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.7331196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.7332001Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.7332824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.7333756Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.7334611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.7335468Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.7336226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.7336911Z return self.act(input) 2025-12-04T09:37:53.7337113Z 2025-12-04T09:37:53.7337247Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7337623Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7338042Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7338410Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7338759Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7339119Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7339492Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7339845Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7340256Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7340955Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7341575Z res = mod(**inputs) 2025-12-04T09:37:53.7342284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7343124Z outputs = self.bert( 2025-12-04T09:37:53.7343838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7344681Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7345434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7346206Z layer_outputs = layer_module( 2025-12-04T09:37:53.7347036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7348108Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7348891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7349672Z return func(*args, **kwargs) 2025-12-04T09:37:53.7350430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.7351312Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.7352109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7352891Z return func(*args, **kwargs) 2025-12-04T09:37:53.7353635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.7354440Z self_outputs = self.self( 2025-12-04T09:37:53.7355280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7356052Z return func(*args, **kwargs) 2025-12-04T09:37:53.7356809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.7357737Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.7358107Z 2025-12-04T09:37:53.7358256Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7358627Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7359064Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7359791Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7360540Z res = mod(**inputs) 2025-12-04T09:37:53.7361248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7362027Z outputs = self.bert( 2025-12-04T09:37:53.7362746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7363523Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7364282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7365064Z layer_outputs = layer_module( 2025-12-04T09:37:53.7365740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7366470Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7367228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7367979Z return func(*args, **kwargs) 2025-12-04T09:37:53.7368709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.7369506Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.7370317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.7371125Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.7372854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.7373786Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.7374719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.7375558Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.7376319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.7377011Z return self.act(input) 2025-12-04T09:37:53.7377203Z 2025-12-04T09:37:53.7377346Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7377707Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7378071Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7378435Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7378791Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7379154Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7379518Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7379873Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7380292Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7380994Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7381629Z res = mod(**inputs) 2025-12-04T09:37:53.7382326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7383097Z outputs = self.bert( 2025-12-04T09:37:53.7383823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7384604Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7385359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7386135Z layer_outputs = layer_module( 2025-12-04T09:37:53.7386924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7387836Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7388620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7389402Z return func(*args, **kwargs) 2025-12-04T09:37:53.7390177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 558, in forward 2025-12-04T09:37:53.7390987Z self_attention_outputs = self.attention( 2025-12-04T09:37:53.7391777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7392557Z return func(*args, **kwargs) 2025-12-04T09:37:53.7393308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 488, in forward 2025-12-04T09:37:53.7394098Z self_outputs = self.self( 2025-12-04T09:37:53.7394839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7395623Z return func(*args, **kwargs) 2025-12-04T09:37:53.7396367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 413, in forward 2025-12-04T09:37:53.7397278Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:37:53.7397652Z 2025-12-04T09:37:53.7397799Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7398180Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7398626Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7399471Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7400192Z res = mod(**inputs) 2025-12-04T09:37:53.7401456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1280, in forward 2025-12-04T09:37:53.7402287Z outputs = self.bert( 2025-12-04T09:37:53.7403173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1000, in forward 2025-12-04T09:37:53.7403995Z encoder_outputs = self.encoder( 2025-12-04T09:37:53.7404791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 650, in forward 2025-12-04T09:37:53.7405611Z layer_outputs = layer_module( 2025-12-04T09:37:53.7406330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:37:53.7407083Z return super().__call__(*args, **kwargs) 2025-12-04T09:37:53.7407887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:37:53.7408694Z return func(*args, **kwargs) 2025-12-04T09:37:53.7409542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 588, in forward 2025-12-04T09:37:53.7410527Z layer_output = apply_chunking_to_forward( 2025-12-04T09:37:53.7411437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:37:53.7412292Z return forward_fn(*input_tensors) 2025-12-04T09:37:53.7413146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 596, in feed_forward_chunk 2025-12-04T09:37:53.7414128Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:37:53.7415037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 513, in forward 2025-12-04T09:37:53.7415930Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:37:53.7416723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:37:53.7417445Z return self.act(input) 2025-12-04T09:37:53.7417651Z 2025-12-04T09:37:53.7417816Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7418210Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7418611Z cudagraph partition due to non gpu ops 2025-12-04T09:37:53.7419054Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:37:53.7419798Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:37:53.7420451Z res = mod(**inputs) 2025-12-04T09:37:53.7421199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1300, in forward 2025-12-04T09:37:53.7422357Z masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:37:53.7422855Z 2025-12-04T09:38:00.4534895Z Compilation time (from dynamo_timed): 20.916462164 2025-12-04T09:38:00.4656696Z pass 2025-12-04T09:38:00.4657322Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:38:00.4659047Z TIMING: _recursive_pre_grad_passes:0.045 _recursive_joint_graph_passes:0.89165 _recursive_post_grad_passes:0.08858 async_compile.wait:1.1351 code_gen:6.25805 inductor_compile:10.28905 backend_compile:16.70215 gc:0.00017 entire_frame_compile:20.91646 total_wall_time:20.91646 2025-12-04T09:38:00.4661072Z STATS: call_* op count: 289 | FakeTensorMode.__torch_dispatch__:18896 | FakeTensor.__torch_dispatch__:3504 | ProxyTorchDispatchMode.__torch_dispatch__:2792 2025-12-04T09:38:00.4662121Z Dynamo produced 1 graphs covering 289 ops with 0 graph breaks (0 unique) 2025-12-04T09:38:03.2636887Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:38:03.2638950Z import pynvml # type: ignore[import] 2025-12-04T09:38:08.2490084Z 2025-12-04T09:38:43.7389760Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:38:43.7391133Z loading model: 0it [00:35, ?it/s] 2025-12-04T09:38:43.7438215Z cpu eval BlenderbotForCausalLM 2025-12-04T09:38:43.7715639Z Compilation time (from dynamo_timed): 0 2025-12-04T09:38:43.7716071Z pass_due_to_skip 2025-12-04T09:38:43.7717815Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:38:43.7718475Z TIMING: total_wall_time:0 2025-12-04T09:38:43.7718788Z STATS: call_* op count: 0 2025-12-04T09:38:43.7719264Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2025-12-04T09:38:46.3352564Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:38:46.3354611Z import pynvml # type: ignore[import] 2025-12-04T09:38:51.3142040Z 2025-12-04T09:39:07.3505067Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:39:07.3505494Z loading model: 0it [00:16, ?it/s] 2025-12-04T09:39:07.3552885Z cpu eval DebertaV2ForMaskedLM 2025-12-04T09:39:07.3828961Z Compilation time (from dynamo_timed): 0 2025-12-04T09:39:07.3829539Z pass_due_to_skip 2025-12-04T09:39:07.3830272Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:07.3830932Z TIMING: total_wall_time:0 2025-12-04T09:39:07.3831259Z STATS: call_* op count: 0 2025-12-04T09:39:07.3831725Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2025-12-04T09:39:09.7632378Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:39:09.7634241Z import pynvml # type: ignore[import] 2025-12-04T09:39:14.7130025Z 2025-12-04T09:39:16.0515154Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:39:16.0515594Z loading model: 0it [00:01, ?it/s] 2025-12-04T09:39:16.0525695Z cpu eval DistilBertForMaskedLM 2025-12-04T09:39:16.2685090Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:16.3516835Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:16.4336760Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:26.0380614Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0381204Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0381610Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0381990Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0382392Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0382804Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0383201Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0383577Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0383969Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0384356Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0384730Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0385119Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0385513Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0385944Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0387225Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0388338Z res = mod(**inputs) 2025-12-04T09:39:26.0389185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0390082Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0391093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0392002Z return self.transformer( 2025-12-04T09:39:26.0392909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0393816Z layer_outputs = layer_module( 2025-12-04T09:39:26.0394551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0395329Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0396214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:26.0397127Z sa_output = self.attention( 2025-12-04T09:39:26.0397987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:26.0398998Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:26.0399379Z 2025-12-04T09:39:26.0399527Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0399934Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0400389Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0401565Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0402252Z res = mod(**inputs) 2025-12-04T09:39:26.0403090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0404008Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0404875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0405775Z return self.transformer( 2025-12-04T09:39:26.0406632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0407517Z layer_outputs = layer_module( 2025-12-04T09:39:26.0408253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0409031Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0409933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:26.0410893Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:26.0411869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:26.0413047Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:26.0414275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:26.0415095Z return forward_fn(*input_tensors) 2025-12-04T09:39:26.0415960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:26.0416843Z x = self.activation(x) 2025-12-04T09:39:26.0417500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:26.0418199Z return self.act(input) 2025-12-04T09:39:26.0418415Z 2025-12-04T09:39:26.0418617Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0419160Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0419549Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0419936Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0420304Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0420821Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0421209Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0421580Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0422019Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0422745Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0423389Z res = mod(**inputs) 2025-12-04T09:39:26.0424192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0425068Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0425917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0426881Z return self.transformer( 2025-12-04T09:39:26.0427903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0428803Z layer_outputs = layer_module( 2025-12-04T09:39:26.0429525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0430272Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0431173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:26.0432072Z sa_output = self.attention( 2025-12-04T09:39:26.0432914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:26.0433931Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:26.0434324Z 2025-12-04T09:39:26.0434469Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0434876Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0435320Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0436070Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0436747Z res = mod(**inputs) 2025-12-04T09:39:26.0437561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0438466Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0439449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0440320Z return self.transformer( 2025-12-04T09:39:26.0441125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0441993Z layer_outputs = layer_module( 2025-12-04T09:39:26.0442699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0443453Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0444316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:26.0445258Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:26.0446195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:26.0447329Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:26.0448425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:26.0449711Z return forward_fn(*input_tensors) 2025-12-04T09:39:26.0450698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:26.0451583Z x = self.activation(x) 2025-12-04T09:39:26.0452261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:26.0452995Z return self.act(input) 2025-12-04T09:39:26.0453201Z 2025-12-04T09:39:26.0453360Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0453753Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0454159Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0454616Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0454996Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0455389Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0455791Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0456168Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0456614Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0457365Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0458050Z res = mod(**inputs) 2025-12-04T09:39:26.0458857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0459780Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0460657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0461547Z return self.transformer( 2025-12-04T09:39:26.0462489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0463374Z layer_outputs = layer_module( 2025-12-04T09:39:26.0464078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0464829Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0465688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:26.0466565Z sa_output = self.attention( 2025-12-04T09:39:26.0467707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:26.0468714Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:26.0469110Z 2025-12-04T09:39:26.0469255Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0469661Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0470116Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0470854Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0471543Z res = mod(**inputs) 2025-12-04T09:39:26.0472374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0473266Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0474154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0475060Z return self.transformer( 2025-12-04T09:39:26.0475914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0476794Z layer_outputs = layer_module( 2025-12-04T09:39:26.0477529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0478551Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0479407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:26.0480348Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:26.0481346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:26.0482487Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:26.0483568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:26.0484395Z return forward_fn(*input_tensors) 2025-12-04T09:39:26.0485257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:26.0486130Z x = self.activation(x) 2025-12-04T09:39:26.0486764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:26.0487465Z return self.act(input) 2025-12-04T09:39:26.0487666Z 2025-12-04T09:39:26.0487824Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0488217Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0488592Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0488986Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0489371Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0489742Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0490126Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0490509Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0490928Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0491656Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0492320Z res = mod(**inputs) 2025-12-04T09:39:26.0493103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0493977Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0494840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0495713Z return self.transformer( 2025-12-04T09:39:26.0496520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0497391Z layer_outputs = layer_module( 2025-12-04T09:39:26.0498098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0498842Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0499703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:26.0500576Z sa_output = self.attention( 2025-12-04T09:39:26.0501991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:26.0502988Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:26.0503385Z 2025-12-04T09:39:26.0503532Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0503938Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0504387Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0505112Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0505786Z res = mod(**inputs) 2025-12-04T09:39:26.0506716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0507762Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0508642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0509537Z return self.transformer( 2025-12-04T09:39:26.0510475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0519064Z layer_outputs = layer_module( 2025-12-04T09:39:26.0519805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0520579Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0521478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:26.0522429Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:26.0523395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:26.0524663Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:26.0525757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:26.0526576Z return forward_fn(*input_tensors) 2025-12-04T09:39:26.0527436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:26.0528305Z x = self.activation(x) 2025-12-04T09:39:26.0528945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:26.0529648Z return self.act(input) 2025-12-04T09:39:26.0529863Z 2025-12-04T09:39:26.0530005Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0530407Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0530783Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0531171Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0531560Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0531940Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0532328Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0532714Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0533135Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0533854Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0534503Z res = mod(**inputs) 2025-12-04T09:39:26.0535299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0536164Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0537012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0537890Z return self.transformer( 2025-12-04T09:39:26.0538702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0539580Z layer_outputs = layer_module( 2025-12-04T09:39:26.0540287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0541035Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0541896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:26.0542769Z sa_output = self.attention( 2025-12-04T09:39:26.0543601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:26.0544680Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:26.0545051Z 2025-12-04T09:39:26.0545191Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0545590Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0546094Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0546898Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0547729Z res = mod(**inputs) 2025-12-04T09:39:26.0548561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0549467Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0550335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0551239Z return self.transformer( 2025-12-04T09:39:26.0552086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0552970Z layer_outputs = layer_module( 2025-12-04T09:39:26.0553693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0554464Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0555364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:26.0556322Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:26.0557288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:26.0558466Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:26.0559686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:26.0560505Z return forward_fn(*input_tensors) 2025-12-04T09:39:26.0561374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:26.0562239Z x = self.activation(x) 2025-12-04T09:39:26.0562876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:26.0563575Z return self.act(input) 2025-12-04T09:39:26.0563788Z 2025-12-04T09:39:26.0563925Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0564312Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0564683Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0565069Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0565453Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0565823Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0566214Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0566597Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0567023Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0567753Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0568406Z res = mod(**inputs) 2025-12-04T09:39:26.0569205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0570066Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0570914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0571786Z return self.transformer( 2025-12-04T09:39:26.0572591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0573536Z layer_outputs = layer_module( 2025-12-04T09:39:26.0576577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0577354Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0578333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 466, in forward 2025-12-04T09:39:26.0579230Z sa_output = self.attention( 2025-12-04T09:39:26.0580088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 392, in forward 2025-12-04T09:39:26.0581107Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:26.0581487Z 2025-12-04T09:39:26.0581631Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0582042Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0582503Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0583239Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0593812Z res = mod(**inputs) 2025-12-04T09:39:26.0594726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 814, in forward 2025-12-04T09:39:26.0595633Z dlbrt_output = self.distilbert( 2025-12-04T09:39:26.0596521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 724, in forward 2025-12-04T09:39:26.0597427Z return self.transformer( 2025-12-04T09:39:26.0598285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 531, in forward 2025-12-04T09:39:26.0599262Z layer_outputs = layer_module( 2025-12-04T09:39:26.0599972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:26.0601340Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:26.0602260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 484, in forward 2025-12-04T09:39:26.0603235Z ffn_output = self.ffn(sa_output) # (bs, seq_length, dim) 2025-12-04T09:39:26.0604216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 418, in forward 2025-12-04T09:39:26.0605399Z return apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, input) 2025-12-04T09:39:26.0606515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:39:26.0607376Z return forward_fn(*input_tensors) 2025-12-04T09:39:26.0608275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 422, in ff_chunk 2025-12-04T09:39:26.0609178Z x = self.activation(x) 2025-12-04T09:39:26.0609843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:39:26.0610570Z return self.act(input) 2025-12-04T09:39:26.0610781Z 2025-12-04T09:39:26.0610950Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0611346Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0611748Z cudagraph partition due to non gpu ops 2025-12-04T09:39:26.0612204Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:26.0612955Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:39:26.0613727Z res = mod(**inputs) 2025-12-04T09:39:26.0614532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/distilbert/modeling_distilbert.py", line 831, in forward 2025-12-04T09:39:26.0615669Z mlm_loss = self.mlm_loss_fct(prediction_logits.view(-1, prediction_logits.size(-1)), labels.view(-1)) 2025-12-04T09:39:26.0616351Z 2025-12-04T09:39:31.1053208Z Compilation time (from dynamo_timed): 13.921868263 2025-12-04T09:39:31.1053699Z pass 2025-12-04T09:39:31.1054220Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:31.1057306Z TIMING: _recursive_pre_grad_passes:0.0257 _recursive_joint_graph_passes:0.42927 _recursive_post_grad_passes:0.05981 async_compile.wait:1.14054 code_gen:4.80761 inductor_compile:8.26251 backend_compile:11.53181 gc:0.00016 entire_frame_compile:13.92187 total_wall_time:13.92187 2025-12-04T09:39:31.1059280Z STATS: call_* op count: 153 | FakeTensorMode.__torch_dispatch__:10141 | FakeTensor.__torch_dispatch__:1896 | ProxyTorchDispatchMode.__torch_dispatch__:1495 2025-12-04T09:39:31.1060297Z Dynamo produced 1 graphs covering 153 ops with 0 graph breaks (0 unique) 2025-12-04T09:39:33.7360241Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:39:33.7361885Z import pynvml # type: ignore[import] 2025-12-04T09:39:38.6878166Z 2025-12-04T09:39:40.7281781Z loading model: 0it [00:00, ?it/s]`loss_type=None` was set in the config but it is unrecognized. Using the default loss: `ForCausalLMLoss`. 2025-12-04T09:39:40.7283971Z WARNING:transformers.modeling_utils:`loss_type=None` was set in the config but it is unrecognized. Using the default loss: `ForCausalLMLoss`. 2025-12-04T09:39:40.7768995Z 2025-12-04T09:39:40.7769459Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:39:40.7775075Z cpu eval DistillGPT2 2025-12-04T09:39:41.4598516Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:41.7668120Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:42.0583173Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:39:53.9307072Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9308280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9309252Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9310259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:39:53.9311091Z causal_mask = create_causal_mask( 2025-12-04T09:39:53.9311922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:39:53.9313094Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:39:53.9314221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:39:53.9315212Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:39:53.9316305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 655, in find_packed_sequence_indices 2025-12-04T09:39:53.9317460Z first_dummy_value = position_ids[:, :1] - 1 # We just need the diff on this first value to be 1 2025-12-04T09:39:53.9318064Z 2025-12-04T09:39:53.9318241Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9318806Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9319257Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9320231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9321091Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9321995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:39:53.9323197Z causal_mask = create_causal_mask( 2025-12-04T09:39:53.9324209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:39:53.9325341Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:39:53.9326534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:39:53.9327589Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:39:53.9328531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 656, in find_packed_sequence_indices 2025-12-04T09:39:53.9329646Z position_diff = torch.diff(position_ids, prepend=first_dummy_value, dim=-1) 2025-12-04T09:39:53.9330091Z 2025-12-04T09:39:53.9330291Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9330740Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9331607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9332505Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9333389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:39:53.9334274Z causal_mask = create_causal_mask( 2025-12-04T09:39:53.9335129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:39:53.9336189Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:39:53.9337269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:39:53.9338277Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:39:53.9339236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:39:53.9340235Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:39:53.9340591Z 2025-12-04T09:39:53.9340787Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9341696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9347517Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9348349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:39:53.9349184Z causal_mask = create_causal_mask( 2025-12-04T09:39:53.9349975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:39:53.9351070Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:39:53.9352192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:39:53.9353167Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:39:53.9354146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:39:53.9355073Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:39:53.9355403Z 2025-12-04T09:39:53.9355598Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9356490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9357344Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9358299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9359097Z outputs = block( 2025-12-04T09:39:53.9359949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9360707Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9361480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9362265Z return func(*args, **kwargs) 2025-12-04T09:39:53.9363029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:53.9363858Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:53.9364645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9365437Z return func(*args, **kwargs) 2025-12-04T09:39:53.9366201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:53.9367033Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:53.9367964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:53.9368972Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:53.9369342Z 2025-12-04T09:39:53.9369501Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9369883Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9370269Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9370657Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9371079Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9371946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9372776Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9373589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9374354Z outputs = block( 2025-12-04T09:39:53.9375007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9375754Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9376524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9377303Z return func(*args, **kwargs) 2025-12-04T09:39:53.9378066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:53.9378891Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:53.9379685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9380464Z return func(*args, **kwargs) 2025-12-04T09:39:53.9381227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:53.9382075Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:53.9382993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:53.9383998Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:53.9384370Z 2025-12-04T09:39:53.9384526Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9384904Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9385347Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9386282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9387389Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9388290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9389107Z outputs = block( 2025-12-04T09:39:53.9389776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9390546Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9391338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9392140Z return func(*args, **kwargs) 2025-12-04T09:39:53.9392926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:53.9393813Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:53.9394673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:53.9395506Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:53.9396250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:53.9397229Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:53.9397735Z 2025-12-04T09:39:53.9397880Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9398289Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9398688Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9399184Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9399567Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9399949Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9400378Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9401879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9402744Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9403600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9409282Z outputs = block( 2025-12-04T09:39:53.9409971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9410750Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9411563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9412358Z return func(*args, **kwargs) 2025-12-04T09:39:53.9413147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:53.9414012Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:53.9414826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9415639Z return func(*args, **kwargs) 2025-12-04T09:39:53.9416430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:53.9417298Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:53.9418250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:53.9419291Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:53.9419796Z 2025-12-04T09:39:53.9419951Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9420329Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9420942Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9421815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9422750Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9423551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9424341Z outputs = block( 2025-12-04T09:39:53.9425003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9425758Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9426524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9427592Z return func(*args, **kwargs) 2025-12-04T09:39:53.9428381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:53.9429254Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:53.9430130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:53.9430961Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:53.9431694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:53.9432648Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:53.9433163Z 2025-12-04T09:39:53.9433305Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9433711Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9434093Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9434486Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9434891Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9435290Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9435723Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9436620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9437473Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9438288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9439181Z outputs = block( 2025-12-04T09:39:53.9439829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9440577Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9441344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9442125Z return func(*args, **kwargs) 2025-12-04T09:39:53.9442885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:53.9443696Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:53.9444499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9445280Z return func(*args, **kwargs) 2025-12-04T09:39:53.9446046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:53.9446880Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:53.9447807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:53.9448804Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:53.9449174Z 2025-12-04T09:39:53.9449407Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9449781Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9450223Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9451153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9451961Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9452763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9453544Z outputs = block( 2025-12-04T09:39:53.9454197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9454930Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9455715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9456498Z return func(*args, **kwargs) 2025-12-04T09:39:53.9457241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:53.9458088Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:53.9458937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:53.9459746Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:53.9460440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:53.9461376Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:53.9461874Z 2025-12-04T09:39:53.9462014Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9462405Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9462776Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9463166Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9463546Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9463913Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9464348Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9465217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9466026Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9467017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9468133Z outputs = block( 2025-12-04T09:39:53.9468808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9469570Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9470383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9471205Z return func(*args, **kwargs) 2025-12-04T09:39:53.9471992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:53.9472891Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:53.9473717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9474519Z return func(*args, **kwargs) 2025-12-04T09:39:53.9475285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:53.9476151Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:53.9477113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:53.9478258Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:53.9478643Z 2025-12-04T09:39:53.9478789Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9479200Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9479832Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9480684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9481508Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9482314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9483102Z outputs = block( 2025-12-04T09:39:53.9483742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9484493Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9485277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9486066Z return func(*args, **kwargs) 2025-12-04T09:39:53.9486817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:53.9487675Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:53.9488524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:53.9489316Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:53.9490035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:53.9490976Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:53.9491465Z 2025-12-04T09:39:53.9491620Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9492053Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9492425Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9492812Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9493196Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9493568Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9494061Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9494926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9495747Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9496538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9497312Z outputs = block( 2025-12-04T09:39:53.9497956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9498691Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9499471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9500246Z return func(*args, **kwargs) 2025-12-04T09:39:53.9501566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:53.9502422Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:53.9503253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9504060Z return func(*args, **kwargs) 2025-12-04T09:39:53.9504832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:53.9505701Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:53.9506667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:53.9507964Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:53.9508341Z 2025-12-04T09:39:53.9508487Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9508973Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9509429Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9510319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9511165Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9512001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9512817Z outputs = block( 2025-12-04T09:39:53.9513478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9514252Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9515057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9515860Z return func(*args, **kwargs) 2025-12-04T09:39:53.9516630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:53.9517507Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:53.9518489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:53.9519288Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:53.9520001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:53.9520938Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:53.9521429Z 2025-12-04T09:39:53.9521582Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9521958Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9522344Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9522727Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9523099Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9523481Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9523914Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9524782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9525589Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9526397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9527178Z outputs = block( 2025-12-04T09:39:53.9527812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9528559Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9529464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9530504Z return func(*args, **kwargs) 2025-12-04T09:39:53.9531276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:39:53.9532123Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:39:53.9532951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9533758Z return func(*args, **kwargs) 2025-12-04T09:39:53.9534527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:39:53.9535391Z attn_output, attn_weights = attention_interface( 2025-12-04T09:39:53.9536459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:39:53.9537479Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:39:53.9537945Z 2025-12-04T09:39:53.9538091Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9538494Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9538950Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9539826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward 2025-12-04T09:39:53.9540678Z transformer_outputs = self.transformer( 2025-12-04T09:39:53.9541511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:39:53.9542305Z outputs = block( 2025-12-04T09:39:53.9543076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:39:53.9543828Z return super().__call__(*args, **kwargs) 2025-12-04T09:39:53.9544612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:39:53.9545375Z return func(*args, **kwargs) 2025-12-04T09:39:53.9546134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:39:53.9547081Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:39:53.9548154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:39:53.9548976Z hidden_states = self.act(hidden_states) 2025-12-04T09:39:53.9549716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:39:53.9550696Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:39:53.9551200Z 2025-12-04T09:39:53.9551343Z cudagraph partition due to non gpu ops 2025-12-04T09:39:53.9551800Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:53.9552698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1092, in forward 2025-12-04T09:39:53.9553612Z logits = self.lm_head(hidden_states[:, slice_indices, :]) 2025-12-04T09:39:53.9553954Z 2025-12-04T09:39:59.4331855Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:39:59.4332904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:39:59.4334370Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:39:59.4335431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:39:59.4336573Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:39:59.4337114Z 2025-12-04T09:40:01.0414561Z Compilation time (from dynamo_timed): 17.858973917 2025-12-04T09:40:01.0653705Z pass 2025-12-04T09:40:01.0654638Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:01.0657881Z TIMING: gc:0.00324 entire_frame_compile:17.85897 _recursive_pre_grad_passes:0.03742 _recursive_joint_graph_passes:0.38443 _recursive_post_grad_passes:0.06543 async_compile.wait:2.02727 code_gen:6.22804 inductor_compile:9.42446 backend_compile:12.47463 total_wall_time:17.85897 2025-12-04T09:40:01.0661273Z STATS: call_* op count: 311 | FakeTensorMode.__torch_dispatch__:9916 | FakeTensor.__torch_dispatch__:1939 | ProxyTorchDispatchMode.__torch_dispatch__:980 2025-12-04T09:40:01.0662641Z Dynamo produced 2 graphs covering 311 ops with 2 graph breaks (1 unique) 2025-12-04T09:40:03.7261695Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:40:03.7263413Z import pynvml # type: ignore[import] 2025-12-04T09:40:08.6700595Z 2025-12-04T09:40:08.6713916Z loading model: 0it [00:00, ?it/s]If you want to use `ElectraForCausalLM` as a standalone, add `is_decoder=True.` 2025-12-04T09:40:08.6715111Z WARNING:transformers.models.electra.modeling_electra:If you want to use `ElectraForCausalLM` as a standalone, add `is_decoder=True.` 2025-12-04T09:40:09.0556975Z 2025-12-04T09:40:09.0557467Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:40:09.0583990Z cpu eval ElectraForCausalLM 2025-12-04T09:40:09.2659651Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:09.3705753Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:09.4836529Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:25.6598059Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6598615Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6599381Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6600174Z res = mod(**inputs) 2025-12-04T09:40:25.6601652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6602553Z outputs = self.electra( 2025-12-04T09:40:25.6603373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 787, in forward 2025-12-04T09:40:25.6608786Z hidden_states = self.embeddings_project(hidden_states) 2025-12-04T09:40:25.6609143Z 2025-12-04T09:40:25.6609299Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6609746Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6610151Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6610572Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6610964Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6611425Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6611824Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6612214Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6612615Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6613015Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6613454Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6614208Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6614898Z res = mod(**inputs) 2025-12-04T09:40:25.6615695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6616540Z outputs = self.electra( 2025-12-04T09:40:25.6617360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6618223Z hidden_states = self.encoder( 2025-12-04T09:40:25.6619146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6619971Z layer_outputs = layer_module( 2025-12-04T09:40:25.6620682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6621430Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6622202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6623289Z return func(*args, **kwargs) 2025-12-04T09:40:25.6624093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6625058Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6625887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6626845Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6627920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6628928Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6629885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6630817Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6631633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6632354Z return self.act(input) 2025-12-04T09:40:25.6632583Z 2025-12-04T09:40:25.6632739Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6633142Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6633529Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6633928Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6634334Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6634732Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6635118Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6635517Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6635921Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6636305Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6636699Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6637156Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6637885Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6638567Z res = mod(**inputs) 2025-12-04T09:40:25.6639471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6640313Z outputs = self.electra( 2025-12-04T09:40:25.6641076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6641913Z hidden_states = self.encoder( 2025-12-04T09:40:25.6642717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6643540Z layer_outputs = layer_module( 2025-12-04T09:40:25.6644245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6645003Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6645801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6646575Z return func(*args, **kwargs) 2025-12-04T09:40:25.6647367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6648225Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6649050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6649862Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6650741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6651727Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6652723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6653626Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6654477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6655186Z return self.act(input) 2025-12-04T09:40:25.6655389Z 2025-12-04T09:40:25.6655530Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6655921Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6656304Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6656673Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6657054Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6657435Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6657817Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6658190Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6658570Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6658952Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6659322Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6659761Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6660479Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6661121Z res = mod(**inputs) 2025-12-04T09:40:25.6661879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6662714Z outputs = self.electra( 2025-12-04T09:40:25.6663488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6664308Z hidden_states = self.encoder( 2025-12-04T09:40:25.6665108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6665953Z layer_outputs = layer_module( 2025-12-04T09:40:25.6671988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6672772Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6673584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6674386Z return func(*args, **kwargs) 2025-12-04T09:40:25.6675184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6676061Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6676924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6677777Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6678682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6679694Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6680642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6681664Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6682443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6683157Z return self.act(input) 2025-12-04T09:40:25.6683360Z 2025-12-04T09:40:25.6683519Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6683994Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6684383Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6684774Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6685312Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6685699Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6686090Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6686477Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6686919Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6687306Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6687689Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6688108Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6688830Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6689483Z res = mod(**inputs) 2025-12-04T09:40:25.6690233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6691068Z outputs = self.electra( 2025-12-04T09:40:25.6691841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6692672Z hidden_states = self.encoder( 2025-12-04T09:40:25.6693458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6694285Z layer_outputs = layer_module( 2025-12-04T09:40:25.6695000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6695730Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6696514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6697294Z return func(*args, **kwargs) 2025-12-04T09:40:25.6698073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6698931Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6699772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6700601Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6702094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6703123Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6704076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6705018Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6705810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6706546Z return self.act(input) 2025-12-04T09:40:25.6706907Z 2025-12-04T09:40:25.6707079Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6707472Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6707873Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6708279Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6708667Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6709067Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6709462Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6709861Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6710241Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6710639Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6711032Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6711460Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6712204Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6712877Z res = mod(**inputs) 2025-12-04T09:40:25.6713648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6714675Z outputs = self.electra( 2025-12-04T09:40:25.6715561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6716423Z hidden_states = self.encoder( 2025-12-04T09:40:25.6717227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6718081Z layer_outputs = layer_module( 2025-12-04T09:40:25.6718898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6719650Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6720420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6721204Z return func(*args, **kwargs) 2025-12-04T09:40:25.6722052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6722891Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6723717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6724540Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6725415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6726387Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6727303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6728213Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6729333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6730061Z return self.act(input) 2025-12-04T09:40:25.6730286Z 2025-12-04T09:40:25.6730433Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6730847Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6731233Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6731633Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6732026Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6732407Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6732798Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6733189Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6733583Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6733962Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6734357Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6734805Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6735538Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6736216Z res = mod(**inputs) 2025-12-04T09:40:25.6737009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6737849Z outputs = self.electra( 2025-12-04T09:40:25.6738651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6739504Z hidden_states = self.encoder( 2025-12-04T09:40:25.6740322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6741271Z layer_outputs = layer_module( 2025-12-04T09:40:25.6741976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6742808Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6743588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6744348Z return func(*args, **kwargs) 2025-12-04T09:40:25.6745199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6746054Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6746969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6748005Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6748915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6749933Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6750868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6751799Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6752604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6753332Z return self.act(input) 2025-12-04T09:40:25.6753545Z 2025-12-04T09:40:25.6753690Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6754095Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6754497Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6754883Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6755277Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6755667Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6756046Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6756439Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6756836Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6757213Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6757609Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6758057Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6758907Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6759554Z res = mod(**inputs) 2025-12-04T09:40:25.6760319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6761153Z outputs = self.electra( 2025-12-04T09:40:25.6761915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6762747Z hidden_states = self.encoder( 2025-12-04T09:40:25.6763545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6764380Z layer_outputs = layer_module( 2025-12-04T09:40:25.6765067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6765815Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6766609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6767377Z return func(*args, **kwargs) 2025-12-04T09:40:25.6768169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6769028Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6769863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6770677Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6771552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6772617Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6774954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6775878Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6776665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6777377Z return self.act(input) 2025-12-04T09:40:25.6777579Z 2025-12-04T09:40:25.6777719Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6778119Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6778508Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6778895Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6779259Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6779647Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6780031Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6780398Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6780783Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6781168Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6781534Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6781971Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6782690Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6783343Z res = mod(**inputs) 2025-12-04T09:40:25.6784087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6784930Z outputs = self.electra( 2025-12-04T09:40:25.6785706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6786528Z hidden_states = self.encoder( 2025-12-04T09:40:25.6787701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6788561Z layer_outputs = layer_module( 2025-12-04T09:40:25.6789284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6790038Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6790848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6791833Z return func(*args, **kwargs) 2025-12-04T09:40:25.6792638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6793521Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6794393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6795249Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6796150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6797167Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6798111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6799040Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6799826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6800554Z return self.act(input) 2025-12-04T09:40:25.6811066Z 2025-12-04T09:40:25.6811282Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6811904Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6812314Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6812715Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6813209Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6813688Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6814077Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6814450Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6814836Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6815223Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6815595Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6816037Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6816776Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6817433Z res = mod(**inputs) 2025-12-04T09:40:25.6818202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6819054Z outputs = self.electra( 2025-12-04T09:40:25.6819840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6820669Z hidden_states = self.encoder( 2025-12-04T09:40:25.6821473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6822302Z layer_outputs = layer_module( 2025-12-04T09:40:25.6823010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6823744Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6824533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6825318Z return func(*args, **kwargs) 2025-12-04T09:40:25.6826103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6827241Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6828115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6828966Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6829860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6830882Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6831836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6832772Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6833559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6834301Z return self.act(input) 2025-12-04T09:40:25.6834507Z 2025-12-04T09:40:25.6834671Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6835062Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6835466Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6835865Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6836261Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6836637Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6837033Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6837431Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6837809Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6838204Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6838597Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6839030Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6839869Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6840611Z res = mod(**inputs) 2025-12-04T09:40:25.6841362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6842278Z outputs = self.electra( 2025-12-04T09:40:25.6843061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6843893Z hidden_states = self.encoder( 2025-12-04T09:40:25.6844677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6845513Z layer_outputs = layer_module( 2025-12-04T09:40:25.6846220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6846971Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6847747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6848531Z return func(*args, **kwargs) 2025-12-04T09:40:25.6849328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6850169Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6851003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6851829Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6852710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6853765Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6859704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6860656Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6861461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6862179Z return self.act(input) 2025-12-04T09:40:25.6862403Z 2025-12-04T09:40:25.6862549Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6862953Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6863336Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6863737Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6864134Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6864514Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6864911Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6865309Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6865708Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6866095Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6866494Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6867094Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6867834Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6868516Z res = mod(**inputs) 2025-12-04T09:40:25.6869310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6870165Z outputs = self.electra( 2025-12-04T09:40:25.6870970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6871826Z hidden_states = self.encoder( 2025-12-04T09:40:25.6872651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6873484Z layer_outputs = layer_module( 2025-12-04T09:40:25.6874307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6875078Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6875951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6876742Z return func(*args, **kwargs) 2025-12-04T09:40:25.6877554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6878537Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6879354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6880181Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6881061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6882055Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6882966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6883871Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6884650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6885364Z return self.act(input) 2025-12-04T09:40:25.6885571Z 2025-12-04T09:40:25.6885713Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6886108Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6886495Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6886868Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6887253Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6887640Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6888015Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6888400Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6888785Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6889154Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6889543Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6889979Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6890703Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6891346Z res = mod(**inputs) 2025-12-04T09:40:25.6892114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1533, in forward 2025-12-04T09:40:25.6892951Z outputs = self.electra( 2025-12-04T09:40:25.6893718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 789, in forward 2025-12-04T09:40:25.6894556Z hidden_states = self.encoder( 2025-12-04T09:40:25.6895355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 559, in forward 2025-12-04T09:40:25.6896182Z layer_outputs = layer_module( 2025-12-04T09:40:25.6896876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:25.6897628Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:25.6898411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:25.6899191Z return func(*args, **kwargs) 2025-12-04T09:40:25.6899968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 496, in forward 2025-12-04T09:40:25.6901384Z layer_output = apply_chunking_to_forward( 2025-12-04T09:40:25.6902260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:40:25.6903252Z return forward_fn(*input_tensors) 2025-12-04T09:40:25.6904247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 504, in feed_forward_chunk 2025-12-04T09:40:25.6905268Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:40:25.6906220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 419, in forward 2025-12-04T09:40:25.6907280Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:40:25.6908082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:40:25.6908812Z return self.act(input) 2025-12-04T09:40:25.6909021Z 2025-12-04T09:40:25.6909184Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6909573Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6909980Z cudagraph partition due to non gpu ops 2025-12-04T09:40:25.6910434Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:25.6911161Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:25.6911840Z res = mod(**inputs) 2025-12-04T09:40:25.6912632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/electra/modeling_electra.py", line 1554, in forward 2025-12-04T09:40:25.6913482Z lm_loss = self.loss_function( 2025-12-04T09:40:25.6914249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:40:25.6915269Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:40:25.6916384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:40:25.6922326Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:40:25.6922889Z 2025-12-04T09:40:32.1337831Z Compilation time (from dynamo_timed): 21.768744442 2025-12-04T09:40:32.1430440Z pass 2025-12-04T09:40:32.1431100Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:32.1432818Z TIMING: _recursive_pre_grad_passes:0.0506 _recursive_joint_graph_passes:0.73811 _recursive_post_grad_passes:0.09005 async_compile.wait:1.18188 code_gen:5.89606 inductor_compile:10.25047 backend_compile:17.26709 gc:0.00039 entire_frame_compile:21.76874 total_wall_time:21.76874 2025-12-04T09:40:32.1434782Z STATS: call_* op count: 377 | FakeTensorMode.__torch_dispatch__:20521 | FakeTensor.__torch_dispatch__:3510 | ProxyTorchDispatchMode.__torch_dispatch__:3375 2025-12-04T09:40:32.1435837Z Dynamo produced 1 graphs covering 377 ops with 0 graph breaks (0 unique) 2025-12-04T09:40:34.9223340Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:40:34.9225076Z import pynvml # type: ignore[import] 2025-12-04T09:40:39.9028046Z 2025-12-04T09:40:42.4927986Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:40:42.4928430Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:40:42.4944481Z cpu eval GPT2ForSequenceClassification 2025-12-04T09:40:44.0522701Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:44.6994433Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:45.2987343Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:40:58.9217538Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9218843Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9219686Z res = mod(**inputs) 2025-12-04T09:40:58.9220721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9221672Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9222662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:40:58.9223655Z causal_mask = create_causal_mask( 2025-12-04T09:40:58.9224479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:40:58.9225704Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:40:58.9227201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:40:58.9228351Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:40:58.9229363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 655, in find_packed_sequence_indices 2025-12-04T09:40:58.9230554Z first_dummy_value = position_ids[:, :1] - 1 # We just need the diff on this first value to be 1 2025-12-04T09:40:58.9231171Z 2025-12-04T09:40:58.9231339Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9231801Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9232702Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9233539Z res = mod(**inputs) 2025-12-04T09:40:58.9234298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1394, in forward 2025-12-04T09:40:58.9235323Z last_non_pad_token = (token_indices * non_pad_mask).argmax(-1) 2025-12-04T09:40:58.9235778Z 2025-12-04T09:40:58.9235979Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9236590Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9237491Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9238385Z res = mod(**inputs) 2025-12-04T09:40:58.9239152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9240098Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9240917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:40:58.9241900Z causal_mask = create_causal_mask( 2025-12-04T09:40:58.9242700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:40:58.9243794Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:40:58.9244907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:40:58.9245954Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:40:58.9247077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 656, in find_packed_sequence_indices 2025-12-04T09:40:58.9248228Z position_diff = torch.diff(position_ids, prepend=first_dummy_value, dim=-1) 2025-12-04T09:40:58.9248671Z 2025-12-04T09:40:58.9248820Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9249371Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9250183Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9250896Z res = mod(**inputs) 2025-12-04T09:40:58.9251863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9252993Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9254076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:40:58.9254895Z causal_mask = create_causal_mask( 2025-12-04T09:40:58.9255839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:40:58.9257059Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:40:58.9258311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:40:58.9259340Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:40:58.9260363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:40:58.9261433Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:40:58.9261758Z 2025-12-04T09:40:58.9262011Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9262788Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9263558Z res = mod(**inputs) 2025-12-04T09:40:58.9264312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9265265Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9266166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:40:58.9267071Z causal_mask = create_causal_mask( 2025-12-04T09:40:58.9268023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:40:58.9269092Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:40:58.9270220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:40:58.9271188Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:40:58.9272164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:40:58.9273074Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:40:58.9273421Z 2025-12-04T09:40:58.9273617Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9274359Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9275026Z res = mod(**inputs) 2025-12-04T09:40:58.9275789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9276639Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9277480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9278272Z outputs = block( 2025-12-04T09:40:58.9278940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9279812Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9280587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9281366Z return func(*args, **kwargs) 2025-12-04T09:40:58.9282122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9283039Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9283827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9284610Z return func(*args, **kwargs) 2025-12-04T09:40:58.9285431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9286280Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9287199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9288204Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9288573Z 2025-12-04T09:40:58.9288727Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9289105Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9289500Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9289895Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9290333Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9291042Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9291705Z res = mod(**inputs) 2025-12-04T09:40:58.9292439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9293253Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9294066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9294860Z outputs = block( 2025-12-04T09:40:58.9295513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9296251Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9297036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9297827Z return func(*args, **kwargs) 2025-12-04T09:40:58.9298577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9299402Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9300233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9311756Z return func(*args, **kwargs) 2025-12-04T09:40:58.9312532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9313404Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9314363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9315403Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9315783Z 2025-12-04T09:40:58.9315927Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9316330Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9316785Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9317509Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9318181Z res = mod(**inputs) 2025-12-04T09:40:58.9319027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9319853Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9320645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9321432Z outputs = block( 2025-12-04T09:40:58.9322089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9322981Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9323768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9324635Z return func(*args, **kwargs) 2025-12-04T09:40:58.9325395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9326237Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9327084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9327899Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9328599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9329541Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9330046Z 2025-12-04T09:40:58.9330186Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9330582Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9330957Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9331340Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9331724Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9332092Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9332523Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9333242Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9333892Z res = mod(**inputs) 2025-12-04T09:40:58.9334606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9335429Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9336236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9337002Z outputs = block( 2025-12-04T09:40:58.9337656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9338401Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9339178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9339938Z return func(*args, **kwargs) 2025-12-04T09:40:58.9340698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9341521Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9342304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9343084Z return func(*args, **kwargs) 2025-12-04T09:40:58.9343838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9344681Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9345594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9346591Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9347090Z 2025-12-04T09:40:58.9347233Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9347802Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9348237Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9348981Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9349655Z res = mod(**inputs) 2025-12-04T09:40:58.9350487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9351335Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9352228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9353042Z outputs = block( 2025-12-04T09:40:58.9353699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9354470Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9355279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9356070Z return func(*args, **kwargs) 2025-12-04T09:40:58.9356855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9357741Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9358724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9359519Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9360230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9361174Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9361660Z 2025-12-04T09:40:58.9361812Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9362187Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9362572Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9363048Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9363691Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9364096Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9364556Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9365292Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9365968Z res = mod(**inputs) 2025-12-04T09:40:58.9366726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9367581Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9368398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9369208Z outputs = block( 2025-12-04T09:40:58.9369885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9370645Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9371456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9372268Z return func(*args, **kwargs) 2025-12-04T09:40:58.9373050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9373890Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9374721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9375525Z return func(*args, **kwargs) 2025-12-04T09:40:58.9376307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9377162Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9378133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9379168Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9379733Z 2025-12-04T09:40:58.9379876Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9380276Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9380716Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9381505Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9382145Z res = mod(**inputs) 2025-12-04T09:40:58.9382872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9383693Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9384480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9385262Z outputs = block( 2025-12-04T09:40:58.9385916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9386748Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9387703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9388507Z return func(*args, **kwargs) 2025-12-04T09:40:58.9389300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9390183Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9391042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9391878Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9392611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9393569Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9394090Z 2025-12-04T09:40:58.9394236Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9394637Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9395034Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9395420Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9395815Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9396205Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9396636Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9397381Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9398058Z res = mod(**inputs) 2025-12-04T09:40:58.9398794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9399737Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9400542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9401980Z outputs = block( 2025-12-04T09:40:58.9402642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9403421Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9404235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9405036Z return func(*args, **kwargs) 2025-12-04T09:40:58.9405804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9406655Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9407478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9408264Z return func(*args, **kwargs) 2025-12-04T09:40:58.9409179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9410049Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9411091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9412114Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9412508Z 2025-12-04T09:40:58.9412653Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9413054Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9413595Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9414317Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9414968Z res = mod(**inputs) 2025-12-04T09:40:58.9415693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9416510Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9417316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9418097Z outputs = block( 2025-12-04T09:40:58.9418730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9419475Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9420256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9421036Z return func(*args, **kwargs) 2025-12-04T09:40:58.9421775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9422626Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9423474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9424284Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9424985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9430627Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9431136Z 2025-12-04T09:40:58.9431295Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9431687Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9432085Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9432481Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9432876Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9433259Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9433703Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9434458Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9435183Z res = mod(**inputs) 2025-12-04T09:40:58.9435947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9436802Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9437635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9438542Z outputs = block( 2025-12-04T09:40:58.9439193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9439939Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9440709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9441488Z return func(*args, **kwargs) 2025-12-04T09:40:58.9442350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9443185Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9444042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9444826Z return func(*args, **kwargs) 2025-12-04T09:40:58.9445583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9446418Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9447352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9448365Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9448734Z 2025-12-04T09:40:58.9448891Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9449274Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9449716Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9450436Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9451097Z res = mod(**inputs) 2025-12-04T09:40:58.9451816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9452653Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9453462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9454222Z outputs = block( 2025-12-04T09:40:58.9454870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9455614Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9456400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9457157Z return func(*args, **kwargs) 2025-12-04T09:40:58.9457923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9458778Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9459604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9460408Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9461117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9462056Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9462541Z 2025-12-04T09:40:58.9462681Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9463084Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9463473Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9463842Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9464227Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9464619Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9465051Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9465758Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9466411Z res = mod(**inputs) 2025-12-04T09:40:58.9467439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9468276Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9469109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9470014Z outputs = block( 2025-12-04T09:40:58.9470687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9471448Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9472313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9473119Z return func(*args, **kwargs) 2025-12-04T09:40:58.9473880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9474726Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9475546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9476350Z return func(*args, **kwargs) 2025-12-04T09:40:58.9477116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9477985Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9479048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9480052Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9480419Z 2025-12-04T09:40:58.9480557Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9480948Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9481388Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9482090Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9482742Z res = mod(**inputs) 2025-12-04T09:40:58.9483465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9484291Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9485083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9485865Z outputs = block( 2025-12-04T09:40:58.9486517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9487249Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9488102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9489180Z return func(*args, **kwargs) 2025-12-04T09:40:58.9489958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9490822Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9491693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9492535Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9493253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9494226Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9494744Z 2025-12-04T09:40:58.9494889Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9495378Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9495765Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9496165Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9496568Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9496967Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9497402Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9498149Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9498909Z res = mod(**inputs) 2025-12-04T09:40:58.9499643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9500495Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9501843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9502661Z outputs = block( 2025-12-04T09:40:58.9503319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9504103Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9504913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9505703Z return func(*args, **kwargs) 2025-12-04T09:40:58.9506488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9507450Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9508279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9509075Z return func(*args, **kwargs) 2025-12-04T09:40:58.9509863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9510736Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9511681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9512719Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9513117Z 2025-12-04T09:40:58.9513262Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9513670Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9514119Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9514862Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9515538Z res = mod(**inputs) 2025-12-04T09:40:58.9516288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9517136Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9517972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9518878Z outputs = block( 2025-12-04T09:40:58.9519518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9520275Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9521068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9521852Z return func(*args, **kwargs) 2025-12-04T09:40:58.9522597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9523461Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9524302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9525097Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9525803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9526740Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9527223Z 2025-12-04T09:40:58.9527380Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9527759Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9528241Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9528625Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9528993Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9529374Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9529874Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9530600Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9531238Z res = mod(**inputs) 2025-12-04T09:40:58.9531964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9532786Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9533577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9534360Z outputs = block( 2025-12-04T09:40:58.9535010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9535766Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9536540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9537318Z return func(*args, **kwargs) 2025-12-04T09:40:58.9538078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9538887Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9539685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9540460Z return func(*args, **kwargs) 2025-12-04T09:40:58.9541219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9542049Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9542973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9543978Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9544346Z 2025-12-04T09:40:58.9544503Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9544877Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9545317Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9546034Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9546771Z res = mod(**inputs) 2025-12-04T09:40:58.9547701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9548557Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9549385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9550223Z outputs = block( 2025-12-04T09:40:58.9551014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9551795Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9552585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9553387Z return func(*args, **kwargs) 2025-12-04T09:40:58.9554174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9555056Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9555912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9556742Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9557559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9558594Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9559113Z 2025-12-04T09:40:58.9559257Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9559662Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9560061Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9560440Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9560831Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9561225Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9561658Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9562402Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9563178Z res = mod(**inputs) 2025-12-04T09:40:58.9563912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9564727Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9565538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9566318Z outputs = block( 2025-12-04T09:40:58.9566953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9567700Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9568477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9569247Z return func(*args, **kwargs) 2025-12-04T09:40:58.9569990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9570819Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9571618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9572382Z return func(*args, **kwargs) 2025-12-04T09:40:58.9573149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9573989Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9574919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9575904Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9576286Z 2025-12-04T09:40:58.9576429Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9576823Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9577263Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9577973Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9578633Z res = mod(**inputs) 2025-12-04T09:40:58.9579377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9580190Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9581006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9581795Z outputs = block( 2025-12-04T09:40:58.9582449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9583184Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9583970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9584818Z return func(*args, **kwargs) 2025-12-04T09:40:58.9585560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9586424Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9587611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9588456Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9589178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9590143Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9590653Z 2025-12-04T09:40:58.9590797Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9591201Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9591584Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9591984Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9592376Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9592754Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9593200Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9593945Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9594602Z res = mod(**inputs) 2025-12-04T09:40:58.9595347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9596190Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9597025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9597824Z outputs = block( 2025-12-04T09:40:58.9598502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9599380Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9600153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9601476Z return func(*args, **kwargs) 2025-12-04T09:40:58.9602271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9603118Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9603934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9604735Z return func(*args, **kwargs) 2025-12-04T09:40:58.9605517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9606384Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9607334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9608365Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9608743Z 2025-12-04T09:40:58.9608907Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9609299Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9609759Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9610503Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9611181Z res = mod(**inputs) 2025-12-04T09:40:58.9611916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9612825Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9618639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9619572Z outputs = block( 2025-12-04T09:40:58.9631618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9632502Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9633503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9634324Z return func(*args, **kwargs) 2025-12-04T09:40:58.9635121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9635995Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9636872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9637717Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9638445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9639528Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9640034Z 2025-12-04T09:40:58.9640182Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9640579Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9640950Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9641340Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9641728Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9642097Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9642534Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9643258Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9643915Z res = mod(**inputs) 2025-12-04T09:40:58.9644630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9645467Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9646280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9647053Z outputs = block( 2025-12-04T09:40:58.9647709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9648459Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9649245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9650010Z return func(*args, **kwargs) 2025-12-04T09:40:58.9650769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9651595Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9652397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9653161Z return func(*args, **kwargs) 2025-12-04T09:40:58.9653925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9654769Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9655685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9656688Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9657076Z 2025-12-04T09:40:58.9657217Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9657613Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9658040Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9658761Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9659495Z res = mod(**inputs) 2025-12-04T09:40:58.9660217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9661139Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9661951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9662737Z outputs = block( 2025-12-04T09:40:58.9663371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9664125Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9664914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9665682Z return func(*args, **kwargs) 2025-12-04T09:40:58.9666444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9667591Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9668484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9669308Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9670049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9671023Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9671521Z 2025-12-04T09:40:58.9671681Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9672070Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9672472Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9672876Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9673264Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9673660Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9674109Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9674840Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9675593Z res = mod(**inputs) 2025-12-04T09:40:58.9681361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9682223Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9683041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9683851Z outputs = block( 2025-12-04T09:40:58.9684527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9685283Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9686100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9686910Z return func(*args, **kwargs) 2025-12-04T09:40:58.9687823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:40:58.9688628Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:40:58.9689431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9690211Z return func(*args, **kwargs) 2025-12-04T09:40:58.9690967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:40:58.9691794Z attn_output, attn_weights = attention_interface( 2025-12-04T09:40:58.9692725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:40:58.9693835Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:40:58.9694203Z 2025-12-04T09:40:58.9694344Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9694821Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9695263Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9695984Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9696628Z res = mod(**inputs) 2025-12-04T09:40:58.9697356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:40:58.9698184Z transformer_outputs = self.transformer( 2025-12-04T09:40:58.9698974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:40:58.9699764Z outputs = block( 2025-12-04T09:40:58.9700424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:40:58.9701750Z return super().__call__(*args, **kwargs) 2025-12-04T09:40:58.9702724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:40:58.9703533Z return func(*args, **kwargs) 2025-12-04T09:40:58.9704320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:40:58.9705204Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:40:58.9706074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:40:58.9707035Z hidden_states = self.act(hidden_states) 2025-12-04T09:40:58.9707780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:40:58.9708749Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:40:58.9709272Z 2025-12-04T09:40:58.9709420Z cudagraph partition due to non gpu ops 2025-12-04T09:40:58.9709884Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9710629Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9711291Z res = mod(**inputs) 2025-12-04T09:40:58.9712043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1379, in forward 2025-12-04T09:40:58.9712883Z logits = self.score(hidden_states) 2025-12-04T09:40:58.9713137Z 2025-12-04T09:40:58.9713344Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9714070Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9714745Z res = mod(**inputs) 2025-12-04T09:40:58.9715496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1422, in forward 2025-12-04T09:40:58.9716450Z loss = loss_fct(pooled_logits.view(-1, self.num_labels), labels.view(-1)) 2025-12-04T09:40:58.9716879Z 2025-12-04T09:40:58.9717078Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:40:58.9717816Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:40:58.9718596Z res = mod(**inputs) 2025-12-04T09:40:58.9719310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1422, in forward 2025-12-04T09:40:58.9720247Z loss = loss_fct(pooled_logits.view(-1, self.num_labels), labels.view(-1)) 2025-12-04T09:40:58.9720648Z 2025-12-04T09:41:14.8035006Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8035825Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8036856Z res = mod(**inputs) 2025-12-04T09:41:14.8037633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8038799Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8039632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:41:14.8040475Z causal_mask = create_causal_mask( 2025-12-04T09:41:14.8041272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:41:14.8042362Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:41:14.8043480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:41:14.8044457Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:41:14.8045637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 655, in find_packed_sequence_indices 2025-12-04T09:41:14.8046972Z first_dummy_value = position_ids[:, :1] - 1 # We just need the diff on this first value to be 1 2025-12-04T09:41:14.8047546Z 2025-12-04T09:41:14.8047702Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8048241Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8049164Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8049953Z res = mod(**inputs) 2025-12-04T09:41:14.8050796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1394, in forward 2025-12-04T09:41:14.8051896Z last_non_pad_token = (token_indices * non_pad_mask).argmax(-1) 2025-12-04T09:41:14.8052307Z 2025-12-04T09:41:14.8052473Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8052988Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8053834Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8054622Z res = mod(**inputs) 2025-12-04T09:41:14.8055430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8056440Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8057377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:41:14.8058310Z causal_mask = create_causal_mask( 2025-12-04T09:41:14.8059139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:41:14.8060392Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:41:14.8061684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:41:14.8062777Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:41:14.8063925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 656, in find_packed_sequence_indices 2025-12-04T09:41:14.8065088Z position_diff = torch.diff(position_ids, prepend=first_dummy_value, dim=-1) 2025-12-04T09:41:14.8065623Z 2025-12-04T09:41:14.8065807Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8066334Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8067633Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8068604Z res = mod(**inputs) 2025-12-04T09:41:14.8069693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8071021Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8071866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:41:14.8072913Z causal_mask = create_causal_mask( 2025-12-04T09:41:14.8074047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:41:14.8075324Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:41:14.8076671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:41:14.8077839Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:41:14.8079140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:41:14.8080165Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:41:14.8080569Z 2025-12-04T09:41:14.8080761Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8081631Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8082402Z res = mod(**inputs) 2025-12-04T09:41:14.8083207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8084038Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8084846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward 2025-12-04T09:41:14.8085638Z causal_mask = create_causal_mask( 2025-12-04T09:41:14.8086410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask 2025-12-04T09:41:14.8087535Z early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments( 2025-12-04T09:41:14.8093472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 740, in _preprocess_mask_arguments 2025-12-04T09:41:14.8094440Z packed_sequence_mask = find_packed_sequence_indices(position_ids) 2025-12-04T09:41:14.8095414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/masking_utils.py", line 657, in find_packed_sequence_indices 2025-12-04T09:41:14.8096340Z packed_sequence_mask = (position_diff != 1).cumsum(-1) 2025-12-04T09:41:14.8096670Z 2025-12-04T09:41:14.8096881Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8097610Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8098278Z res = mod(**inputs) 2025-12-04T09:41:14.8099032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8099891Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8100709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8101987Z outputs = block( 2025-12-04T09:41:14.8102664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8103420Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8104234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8105040Z return func(*args, **kwargs) 2025-12-04T09:41:14.8105827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8106784Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8107801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8108603Z return func(*args, **kwargs) 2025-12-04T09:41:14.8109509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8110388Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8111355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8112399Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8112783Z 2025-12-04T09:41:14.8112930Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8113338Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8113732Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8114115Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8114569Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8115312Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8115989Z res = mod(**inputs) 2025-12-04T09:41:14.8116730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8117581Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8118517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8119287Z outputs = block( 2025-12-04T09:41:14.8119939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8120687Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8121467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8122235Z return func(*args, **kwargs) 2025-12-04T09:41:14.8122991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8123815Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8124610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8125369Z return func(*args, **kwargs) 2025-12-04T09:41:14.8126125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8126965Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8127882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8128884Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8129271Z 2025-12-04T09:41:14.8129412Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8129800Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8130223Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8130948Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8131603Z res = mod(**inputs) 2025-12-04T09:41:14.8132319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8133152Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8133957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8134740Z outputs = block( 2025-12-04T09:41:14.8135381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8136211Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8136999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8137780Z return func(*args, **kwargs) 2025-12-04T09:41:14.8138585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8139434Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8140281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8141072Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8141789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8142739Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8143234Z 2025-12-04T09:41:14.8143390Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8143772Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8144159Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8144548Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8144924Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8145309Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8145747Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8146452Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8147391Z res = mod(**inputs) 2025-12-04T09:41:14.8148151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8149006Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8149862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8150816Z outputs = block( 2025-12-04T09:41:14.8151490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8152271Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8153128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8153937Z return func(*args, **kwargs) 2025-12-04T09:41:14.8154724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8155560Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8156383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8157183Z return func(*args, **kwargs) 2025-12-04T09:41:14.8157969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8158820Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8159779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8160813Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8161193Z 2025-12-04T09:41:14.8161353Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8161738Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8162303Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8163024Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8163658Z res = mod(**inputs) 2025-12-04T09:41:14.8164380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8165294Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8166084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8166932Z outputs = block( 2025-12-04T09:41:14.8167585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8168333Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8169100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8169886Z return func(*args, **kwargs) 2025-12-04T09:41:14.8170644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8171499Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8172336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8173146Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8173861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8174790Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8175288Z 2025-12-04T09:41:14.8175429Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8175822Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8176205Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8176576Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8176961Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8177344Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8177763Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8178489Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8179144Z res = mod(**inputs) 2025-12-04T09:41:14.8179871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8180687Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8181491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8182270Z outputs = block( 2025-12-04T09:41:14.8182905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8183652Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8184432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8185206Z return func(*args, **kwargs) 2025-12-04T09:41:14.8185953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8186872Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8187871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8188655Z return func(*args, **kwargs) 2025-12-04T09:41:14.8189439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8190306Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8191259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8192277Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8192671Z 2025-12-04T09:41:14.8192815Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8193299Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8193750Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8194479Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8195214Z res = mod(**inputs) 2025-12-04T09:41:14.8195963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8196797Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8197624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8198431Z outputs = block( 2025-12-04T09:41:14.8199100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8199943Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8201098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8202157Z return func(*args, **kwargs) 2025-12-04T09:41:14.8202936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8203821Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8204693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8205531Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8206254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8207222Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8207725Z 2025-12-04T09:41:14.8207883Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8208291Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8208675Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8209075Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8209466Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8209848Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8210309Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8211054Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8211711Z res = mod(**inputs) 2025-12-04T09:41:14.8212529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8213486Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8214326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8215128Z outputs = block( 2025-12-04T09:41:14.8215811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8216589Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8217390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8218197Z return func(*args, **kwargs) 2025-12-04T09:41:14.8218983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8219836Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8220645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8221452Z return func(*args, **kwargs) 2025-12-04T09:41:14.8222236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8223251Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8224195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8225427Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8225799Z 2025-12-04T09:41:14.8225957Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8226339Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8226884Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8227813Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8228493Z res = mod(**inputs) 2025-12-04T09:41:14.8229229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8230080Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8230915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8231702Z outputs = block( 2025-12-04T09:41:14.8232376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8233140Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8233944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8234730Z return func(*args, **kwargs) 2025-12-04T09:41:14.8235513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8236386Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8237243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8238087Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8238916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8239861Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8240347Z 2025-12-04T09:41:14.8240489Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8240886Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8241273Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8241655Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8242024Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8242404Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8242841Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8243546Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8244204Z res = mod(**inputs) 2025-12-04T09:41:14.8244934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8245750Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8246562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8247341Z outputs = block( 2025-12-04T09:41:14.8247993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8248723Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8249505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8250282Z return func(*args, **kwargs) 2025-12-04T09:41:14.8251029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8252929Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8253734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8254581Z return func(*args, **kwargs) 2025-12-04T09:41:14.8255331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8256175Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8257101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8258101Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8258468Z 2025-12-04T09:41:14.8258609Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8259000Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8259442Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8260145Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8260802Z res = mod(**inputs) 2025-12-04T09:41:14.8261532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8262362Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8263148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8263929Z outputs = block( 2025-12-04T09:41:14.8264585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8265313Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8266087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8266969Z return func(*args, **kwargs) 2025-12-04T09:41:14.8267917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8268787Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8269661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8270493Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8271230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8272184Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8272707Z 2025-12-04T09:41:14.8272849Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8273254Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8273642Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8274036Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8274426Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8274844Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8280283Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8281034Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8281706Z res = mod(**inputs) 2025-12-04T09:41:14.8282442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8283299Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8284135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8284940Z outputs = block( 2025-12-04T09:41:14.8285596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8286473Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8287456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8288225Z return func(*args, **kwargs) 2025-12-04T09:41:14.8288988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8289820Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8290627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8291391Z return func(*args, **kwargs) 2025-12-04T09:41:14.8292158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8293003Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8293927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8294993Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8295380Z 2025-12-04T09:41:14.8295522Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8295919Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8296346Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8297071Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8297731Z res = mod(**inputs) 2025-12-04T09:41:14.8298448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8299276Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8300087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8301244Z outputs = block( 2025-12-04T09:41:14.8302077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8302854Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8303663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8304464Z return func(*args, **kwargs) 2025-12-04T09:41:14.8305233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8306116Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8307082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8307903Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8308639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8309608Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8310114Z 2025-12-04T09:41:14.8310275Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8310665Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8311063Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8311460Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8311842Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8312236Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8312684Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8313424Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8314077Z res = mod(**inputs) 2025-12-04T09:41:14.8314967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8315823Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8316727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8317536Z outputs = block( 2025-12-04T09:41:14.8318204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8319071Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8319842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8320634Z return func(*args, **kwargs) 2025-12-04T09:41:14.8321393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8322210Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8323008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8323783Z return func(*args, **kwargs) 2025-12-04T09:41:14.8324548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8325377Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8326308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8327309Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8327676Z 2025-12-04T09:41:14.8327829Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8328203Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8328642Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8329363Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8330001Z res = mod(**inputs) 2025-12-04T09:41:14.8330735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8331558Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8332362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8333126Z outputs = block( 2025-12-04T09:41:14.8333770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8334515Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8335281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8336058Z return func(*args, **kwargs) 2025-12-04T09:41:14.8336810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8337742Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8343563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8344401Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8345135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8346090Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8346606Z 2025-12-04T09:41:14.8346865Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8347270Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8347668Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8348176Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8348570Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8348966Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8349404Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8350214Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8350890Z res = mod(**inputs) 2025-12-04T09:41:14.8351643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8352478Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8353305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8354103Z outputs = block( 2025-12-04T09:41:14.8354757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8355535Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8356341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8357144Z return func(*args, **kwargs) 2025-12-04T09:41:14.8357920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8358865Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8359664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8360426Z return func(*args, **kwargs) 2025-12-04T09:41:14.8361184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8362022Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8362958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8363950Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8364334Z 2025-12-04T09:41:14.8364481Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8364879Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8365321Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8366032Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8366683Z res = mod(**inputs) 2025-12-04T09:41:14.8367412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8368227Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8369033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8369828Z outputs = block( 2025-12-04T09:41:14.8370490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8371221Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8372013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8372804Z return func(*args, **kwargs) 2025-12-04T09:41:14.8373548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8374401Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8375251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8376064Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8376760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8377763Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8378250Z 2025-12-04T09:41:14.8378464Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8378854Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8379226Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8379609Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8379994Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8380360Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8380793Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8381514Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8382149Z res = mod(**inputs) 2025-12-04T09:41:14.8382876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8383706Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8384513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8385285Z outputs = block( 2025-12-04T09:41:14.8385933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8386770Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8387728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8388534Z return func(*args, **kwargs) 2025-12-04T09:41:14.8389315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8390171Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8390991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8391795Z return func(*args, **kwargs) 2025-12-04T09:41:14.8392581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8393448Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8394390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8395424Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8395802Z 2025-12-04T09:41:14.8395960Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8396346Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8396796Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8397536Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8398213Z res = mod(**inputs) 2025-12-04T09:41:14.8398949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8399849Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8401137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8401937Z outputs = block( 2025-12-04T09:41:14.8402610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8403382Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8404189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8405183Z return func(*args, **kwargs) 2025-12-04T09:41:14.8405971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8406995Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8407936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8408771Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8409504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8410477Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8410973Z 2025-12-04T09:41:14.8411118Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8411519Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8411913Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8412304Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8412686Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8413195Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8413633Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8414346Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8414997Z res = mod(**inputs) 2025-12-04T09:41:14.8415723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8416531Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8417335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8418118Z outputs = block( 2025-12-04T09:41:14.8418769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8419500Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8420289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8421066Z return func(*args, **kwargs) 2025-12-04T09:41:14.8421829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8422641Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8423441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8424215Z return func(*args, **kwargs) 2025-12-04T09:41:14.8424955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8425795Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8426816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8428017Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8428398Z 2025-12-04T09:41:14.8428544Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8428952Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8429412Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8430143Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8430819Z res = mod(**inputs) 2025-12-04T09:41:14.8431570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8432424Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8433239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8434046Z outputs = block( 2025-12-04T09:41:14.8434802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8435561Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8436431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8437235Z return func(*args, **kwargs) 2025-12-04T09:41:14.8438023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8438893Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8439853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8440672Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8441390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8442315Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8442822Z 2025-12-04T09:41:14.8442965Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8443361Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8443798Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8444167Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8444548Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8444928Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8445364Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8446065Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8446715Z res = mod(**inputs) 2025-12-04T09:41:14.8447442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8448255Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8449059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8449835Z outputs = block( 2025-12-04T09:41:14.8450486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8451219Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8451999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8452779Z return func(*args, **kwargs) 2025-12-04T09:41:14.8453520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8454352Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8455153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8455929Z return func(*args, **kwargs) 2025-12-04T09:41:14.8456679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8457522Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8458447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8459432Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8459815Z 2025-12-04T09:41:14.8459954Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8460344Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8460783Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8461484Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8462240Z res = mod(**inputs) 2025-12-04T09:41:14.8463318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8464164Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8465073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8465879Z outputs = block( 2025-12-04T09:41:14.8466551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8467434Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8468248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8469054Z return func(*args, **kwargs) 2025-12-04T09:41:14.8469833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8481788Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8482688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8483520Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8484239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8485180Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8485668Z 2025-12-04T09:41:14.8485828Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8486207Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8486595Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8486985Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8487356Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8487740Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8488191Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8488899Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8489556Z res = mod(**inputs) 2025-12-04T09:41:14.8490300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8491131Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8491925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8492707Z outputs = block( 2025-12-04T09:41:14.8493360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8494094Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8494878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8495667Z return func(*args, **kwargs) 2025-12-04T09:41:14.8496434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 413, in forward 2025-12-04T09:41:14.8497253Z attn_output, self_attn_weights = self.attn( 2025-12-04T09:41:14.8498054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8498836Z return func(*args, **kwargs) 2025-12-04T09:41:14.8499578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 345, in forward 2025-12-04T09:41:14.8500421Z attn_output, attn_weights = attention_interface( 2025-12-04T09:41:14.8502046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:41:14.8503085Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:41:14.8503637Z 2025-12-04T09:41:14.8503783Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8504192Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8504648Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8505485Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8506151Z res = mod(**inputs) 2025-12-04T09:41:14.8507011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1365, in forward 2025-12-04T09:41:14.8507875Z transformer_outputs = self.transformer( 2025-12-04T09:41:14.8508694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 925, in forward 2025-12-04T09:41:14.8509500Z outputs = block( 2025-12-04T09:41:14.8510172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:14.8510949Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:14.8511747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:41:14.8512555Z return func(*args, **kwargs) 2025-12-04T09:41:14.8513340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 449, in forward 2025-12-04T09:41:14.8514205Z feed_forward_hidden_states = self.mlp(hidden_states) 2025-12-04T09:41:14.8515077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 375, in forward 2025-12-04T09:41:14.8515909Z hidden_states = self.act(hidden_states) 2025-12-04T09:41:14.8516643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:14.8517599Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:14.8518117Z 2025-12-04T09:41:14.8518261Z cudagraph partition due to non gpu ops 2025-12-04T09:41:14.8518822Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8519548Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8520187Z res = mod(**inputs) 2025-12-04T09:41:14.8520915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1379, in forward 2025-12-04T09:41:14.8521726Z logits = self.score(hidden_states) 2025-12-04T09:41:14.8521973Z 2025-12-04T09:41:14.8522161Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8522882Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8523538Z res = mod(**inputs) 2025-12-04T09:41:14.8524263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1422, in forward 2025-12-04T09:41:14.8530276Z loss = loss_fct(pooled_logits.view(-1, self.num_labels), labels.view(-1)) 2025-12-04T09:41:14.8530712Z 2025-12-04T09:41:14.8530906Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:14.8531653Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:14.8532329Z res = mod(**inputs) 2025-12-04T09:41:14.8533067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1422, in forward 2025-12-04T09:41:14.8534043Z loss = loss_fct(pooled_logits.view(-1, self.num_labels), labels.view(-1)) 2025-12-04T09:41:14.8534457Z 2025-12-04T09:41:17.7869985Z Compilation time (from dynamo_timed): 31.338684691 2025-12-04T09:41:17.7870445Z pass 2025-12-04T09:41:17.7870980Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:17.7872984Z TIMING: _recursive_pre_grad_passes:0.11699 _recursive_joint_graph_passes:1.1102 _recursive_post_grad_passes:0.17159 async_compile.wait:1.02138 code_gen:8.85492 inductor_compile:14.49771 backend_compile:24.62176 gc:0.00095 entire_frame_compile:31.33868 total_wall_time:31.33868 2025-12-04T09:41:17.7875077Z STATS: call_* op count: 1126 | FakeTensorMode.__torch_dispatch__:31994 | FakeTensor.__torch_dispatch__:7032 | ProxyTorchDispatchMode.__torch_dispatch__:3409 2025-12-04T09:41:17.7876138Z Dynamo produced 2 graphs covering 1126 ops with 0 graph breaks (0 unique) 2025-12-04T09:41:20.8737448Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:41:20.8739062Z import pynvml # type: ignore[import] 2025-12-04T09:41:25.9000667Z 2025-12-04T09:41:27.6170317Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:41:27.6170894Z loading model: 0it [00:01, ?it/s] 2025-12-04T09:41:27.6180326Z cpu eval GoogleFnet 2025-12-04T09:41:28.2077693Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:28.4671395Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:28.7169004Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:39.2995912Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.2997352Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.2998521Z res = mod(**inputs) 2025-12-04T09:41:39.2999724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3001284Z outputs = self.fnet( 2025-12-04T09:41:39.3002102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3002945Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3003778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3004637Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3005410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3006189Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3007018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3008044Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3008908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3009757Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3010589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3011469Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3011816Z 2025-12-04T09:41:39.3012018Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3012781Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3013574Z res = mod(**inputs) 2025-12-04T09:41:39.3014288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3015081Z outputs = self.fnet( 2025-12-04T09:41:39.3015815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3016924Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3017693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3018524Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3019399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3020146Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3020953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3021809Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3022658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3023461Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3024278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3025150Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3025467Z 2025-12-04T09:41:39.3025664Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3026394Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3027356Z res = mod(**inputs) 2025-12-04T09:41:39.3028104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3028912Z outputs = self.fnet( 2025-12-04T09:41:39.3029672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3030501Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3031292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3032153Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3032933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3033712Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3034526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3035404Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3036275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3037103Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3037933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3038821Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3039257Z 2025-12-04T09:41:39.3039462Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3040172Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3040834Z res = mod(**inputs) 2025-12-04T09:41:39.3041561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3042353Z outputs = self.fnet( 2025-12-04T09:41:39.3043067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3043875Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3044651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3045459Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3046217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3047048Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3047921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3052401Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3053339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3054183Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3055013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3055880Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3056217Z 2025-12-04T09:41:39.3056412Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3057173Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3057831Z res = mod(**inputs) 2025-12-04T09:41:39.3058570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3059382Z outputs = self.fnet( 2025-12-04T09:41:39.3060131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3060935Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3061729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3062574Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3063331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3064097Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3065033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3065873Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3066818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3067832Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3068657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3069541Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3069867Z 2025-12-04T09:41:39.3070062Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3070803Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3071473Z res = mod(**inputs) 2025-12-04T09:41:39.3072209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3073022Z outputs = self.fnet( 2025-12-04T09:41:39.3073775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3074601Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3075383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3076226Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3077003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3077751Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3078717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3079714Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3080556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3081356Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3082226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3083081Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3083392Z 2025-12-04T09:41:39.3083594Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3084292Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3084944Z res = mod(**inputs) 2025-12-04T09:41:39.3085671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3086454Z outputs = self.fnet( 2025-12-04T09:41:39.3087185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3087982Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3088757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3089560Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3090314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3091063Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3091842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3092685Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3093523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3094339Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3095126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3095980Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3096306Z 2025-12-04T09:41:39.3096492Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3097207Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3097846Z res = mod(**inputs) 2025-12-04T09:41:39.3098563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3099350Z outputs = self.fnet( 2025-12-04T09:41:39.3100061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3101494Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3102310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3103154Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3103922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3104690Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3105516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3106399Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3107357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3108199Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3109183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3110050Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3110387Z 2025-12-04T09:41:39.3115589Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3116347Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3117021Z res = mod(**inputs) 2025-12-04T09:41:39.3117749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3118570Z outputs = self.fnet( 2025-12-04T09:41:39.3119325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3120133Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3120931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3121792Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3122568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3123433Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3124231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3125082Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3125921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3126717Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3127519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3128383Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3128700Z 2025-12-04T09:41:39.3128891Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3129610Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3130274Z res = mod(**inputs) 2025-12-04T09:41:39.3130997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3131778Z outputs = self.fnet( 2025-12-04T09:41:39.3132503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3133307Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3134074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3134898Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3135661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3136416Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3137209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3138057Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3138890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3139702Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3140480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3141331Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3141640Z 2025-12-04T09:41:39.3141845Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3142619Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3143268Z res = mod(**inputs) 2025-12-04T09:41:39.3144046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3144839Z outputs = self.fnet( 2025-12-04T09:41:39.3145552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3146357Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3147419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3148275Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3149037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3149805Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3150630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3151486Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3152357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3153197Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3154016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3154881Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3155214Z 2025-12-04T09:41:39.3155408Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3156149Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3156825Z res = mod(**inputs) 2025-12-04T09:41:39.3157561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3158373Z outputs = self.fnet( 2025-12-04T09:41:39.3159230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3160010Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3160781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3161601Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3162349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3163075Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3163870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3164715Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3165538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3166349Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3167144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3167993Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3168303Z 2025-12-04T09:41:39.3168445Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3168884Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3169597Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3170231Z res = mod(**inputs) 2025-12-04T09:41:39.3170950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3171816Z outputs = self.fnet( 2025-12-04T09:41:39.3172546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 511, in forward 2025-12-04T09:41:39.3173567Z embedding_output = self.embeddings( 2025-12-04T09:41:39.3174635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 141, in forward 2025-12-04T09:41:39.3175476Z embeddings = self.projection(embeddings) 2025-12-04T09:41:39.3175757Z 2025-12-04T09:41:39.3175917Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3176356Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3177095Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3177770Z res = mod(**inputs) 2025-12-04T09:41:39.3178501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3179319Z outputs = self.fnet( 2025-12-04T09:41:39.3180068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3180901Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3181682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3182535Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3183311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3184069Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3184885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3185757Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3186625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3187551Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3188378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3189258Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3189580Z 2025-12-04T09:41:39.3189789Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3190516Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3191198Z res = mod(**inputs) 2025-12-04T09:41:39.3191940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3192740Z outputs = self.fnet( 2025-12-04T09:41:39.3193492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3194326Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3195123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3195957Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3196733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3197501Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3198417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3199270Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3200109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3201543Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3202362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3203252Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3203693Z 2025-12-04T09:41:39.3203905Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3204650Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3205311Z res = mod(**inputs) 2025-12-04T09:41:39.3206059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3206880Z outputs = self.fnet( 2025-12-04T09:41:39.3207617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3208444Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3209259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3210107Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3210867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3211640Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3212454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3213311Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3214264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3215070Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3215864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3216707Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3217028Z 2025-12-04T09:41:39.3217216Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3217936Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3218591Z res = mod(**inputs) 2025-12-04T09:41:39.3219290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3220080Z outputs = self.fnet( 2025-12-04T09:41:39.3220818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3221606Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3222384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3223213Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3223967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3224699Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3225497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3226343Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3227471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3228292Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3229121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3230017Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3230337Z 2025-12-04T09:41:39.3230603Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3231065Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3231816Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3232555Z res = mod(**inputs) 2025-12-04T09:41:39.3233287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3234099Z outputs = self.fnet( 2025-12-04T09:41:39.3234851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3235789Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3236661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3237511Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3238288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3239043Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3239868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3240714Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3241617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3242469Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3243339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3244312Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3245199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3246098Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3246895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3247870Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3248479Z 2025-12-04T09:41:39.3248619Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3249010Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3249445Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3250149Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3250802Z res = mod(**inputs) 2025-12-04T09:41:39.3251525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3252311Z outputs = self.fnet( 2025-12-04T09:41:39.3253030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3253829Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3254610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3255430Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3256170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3256913Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3257714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3258547Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3259385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3260289Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3261085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3261985Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3262312Z 2025-12-04T09:41:39.3262499Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3263217Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3263856Z res = mod(**inputs) 2025-12-04T09:41:39.3264576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3265369Z outputs = self.fnet( 2025-12-04T09:41:39.3266100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3266994Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3267967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3268817Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3269603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3270357Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3271187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3272063Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3272915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3273763Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3274589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3275474Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3275792Z 2025-12-04T09:41:39.3275987Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3276738Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3277410Z res = mod(**inputs) 2025-12-04T09:41:39.3278139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3279066Z outputs = self.fnet( 2025-12-04T09:41:39.3279796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3280598Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3281358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3282189Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3282939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3283691Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3284474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3285328Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3286170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3286961Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3287758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3288611Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3288997Z 2025-12-04T09:41:39.3289198Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3289902Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3290550Z res = mod(**inputs) 2025-12-04T09:41:39.3291329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3292117Z outputs = self.fnet( 2025-12-04T09:41:39.3292832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3293629Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3294405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3295210Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3295962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3296712Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3297514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3303914Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3304792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3305633Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3306448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3307447Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3307789Z 2025-12-04T09:41:39.3307936Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3308398Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3309138Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3309811Z res = mod(**inputs) 2025-12-04T09:41:39.3310557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3311374Z outputs = self.fnet( 2025-12-04T09:41:39.3312111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3312928Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3313728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3314557Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3315328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3316097Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3316913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3317744Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3318707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3319529Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3320354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3321294Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3322162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3323030Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3323938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3324879Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3325460Z 2025-12-04T09:41:39.3325602Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3325992Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3326412Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3327129Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3327780Z res = mod(**inputs) 2025-12-04T09:41:39.3328483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3329274Z outputs = self.fnet( 2025-12-04T09:41:39.3330004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3330808Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3331567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3332395Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3333141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3333872Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3334664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3335511Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3336353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3337150Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3337951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3338807Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3339116Z 2025-12-04T09:41:39.3339325Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3340030Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3340687Z res = mod(**inputs) 2025-12-04T09:41:39.3341407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3342182Z outputs = self.fnet( 2025-12-04T09:41:39.3342907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3343714Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3344492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3345302Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3346057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3346906Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3347885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3348755Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3349617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3350456Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3351266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3352248Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3352582Z 2025-12-04T09:41:39.3352779Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3353582Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3354246Z res = mod(**inputs) 2025-12-04T09:41:39.3354992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3355811Z outputs = self.fnet( 2025-12-04T09:41:39.3356547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3357369Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3358169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3359021Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3359788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3365524Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3366362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3367240Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3368087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3368925Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3369745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3370609Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3370941Z 2025-12-04T09:41:39.3371132Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3371877Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3372550Z res = mod(**inputs) 2025-12-04T09:41:39.3373384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3374173Z outputs = self.fnet( 2025-12-04T09:41:39.3374901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3375689Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3376462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3377287Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3378039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3378777Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3379577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3380422Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3381263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3382063Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3382859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3383712Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3384027Z 2025-12-04T09:41:39.3384168Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3384610Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3385328Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3386091Z res = mod(**inputs) 2025-12-04T09:41:39.3386903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3387953Z outputs = self.fnet( 2025-12-04T09:41:39.3388715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3389531Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3390327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3391175Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3391951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3392703Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3393525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3394374Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3395233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3396067Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3396938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3397908Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3398796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3399779Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3400552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3402057Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3402563Z 2025-12-04T09:41:39.3402710Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3403121Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3403573Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3404316Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3404972Z res = mod(**inputs) 2025-12-04T09:41:39.3405716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3406525Z outputs = self.fnet( 2025-12-04T09:41:39.3407260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3408083Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3408882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3409721Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3410485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3411253Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3412076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3412936Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3413906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3414721Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3415522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3416499Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3416826Z 2025-12-04T09:41:39.3417203Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3418004Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3418659Z res = mod(**inputs) 2025-12-04T09:41:39.3419368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3420156Z outputs = self.fnet( 2025-12-04T09:41:39.3420890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3421680Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3422455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3423412Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3424415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3425171Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3426001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3427011Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3427881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3428702Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3429528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3430414Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3430741Z 2025-12-04T09:41:39.3430934Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3431670Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3432349Z res = mod(**inputs) 2025-12-04T09:41:39.3433097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3433899Z outputs = self.fnet( 2025-12-04T09:41:39.3434653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3435478Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3436263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3437104Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3437876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3438755Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3439532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3440378Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3441212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3442018Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3442793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3443648Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3443958Z 2025-12-04T09:41:39.3444156Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3444855Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3445584Z res = mod(**inputs) 2025-12-04T09:41:39.3446297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3447156Z outputs = self.fnet( 2025-12-04T09:41:39.3447870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3448672Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3449443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3450251Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3451003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3451754Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3452558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3453399Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3454237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3455051Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3455850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3456687Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3457010Z 2025-12-04T09:41:39.3457150Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3457590Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3458292Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3458949Z res = mod(**inputs) 2025-12-04T09:41:39.3459668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3460450Z outputs = self.fnet( 2025-12-04T09:41:39.3461170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3461966Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3462739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3463543Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3464295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3465038Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3465832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3466754Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3467784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3488043Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3489116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3490108Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3491032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3491987Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3492794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3493775Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3494396Z 2025-12-04T09:41:39.3494547Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3494963Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3495477Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3496215Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3496889Z res = mod(**inputs) 2025-12-04T09:41:39.3497644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3498457Z outputs = self.fnet( 2025-12-04T09:41:39.3499199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3500033Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3501295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3502160Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3502941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3503728Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3504548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3505403Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3506273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3507242Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3508076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3508946Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3509293Z 2025-12-04T09:41:39.3509488Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3510238Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3510904Z res = mod(**inputs) 2025-12-04T09:41:39.3511655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3512473Z outputs = self.fnet( 2025-12-04T09:41:39.3513227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3514039Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3514847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3515702Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3516475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3517250Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3518083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3519041Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3519863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3520674Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3521473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3522330Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3522642Z 2025-12-04T09:41:39.3522829Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3523696Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3524359Z res = mod(**inputs) 2025-12-04T09:41:39.3525139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3525929Z outputs = self.fnet( 2025-12-04T09:41:39.3526663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3527463Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3528223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3529048Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3529806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3530542Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3531348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3532207Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3533059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3533862Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3534672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3535529Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3535843Z 2025-12-04T09:41:39.3536044Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3536752Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3537404Z res = mod(**inputs) 2025-12-04T09:41:39.3538128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3538901Z outputs = self.fnet( 2025-12-04T09:41:39.3539637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3540437Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3541216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3542022Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3542772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3543520Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3544304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3545168Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3546011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3546937Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3547920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3553866Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3554206Z 2025-12-04T09:41:39.3554356Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3554816Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3555541Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3556224Z res = mod(**inputs) 2025-12-04T09:41:39.3556970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3557857Z outputs = self.fnet( 2025-12-04T09:41:39.3558615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3559500Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3560302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3561130Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3561909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3562673Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3563593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3564412Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3565251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3566080Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3566918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3567862Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3568738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3569611Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3570373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3571318Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3571809Z 2025-12-04T09:41:39.3571971Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3572372Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3572799Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3573536Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3574193Z res = mod(**inputs) 2025-12-04T09:41:39.3574900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3575685Z outputs = self.fnet( 2025-12-04T09:41:39.3576397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3577179Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3577932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3578723Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3579465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3580184Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3580966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3581804Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3582632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3583423Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3584206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3585035Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3585344Z 2025-12-04T09:41:39.3585583Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3586275Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3587018Z res = mod(**inputs) 2025-12-04T09:41:39.3587993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3588787Z outputs = self.fnet( 2025-12-04T09:41:39.3589534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3590345Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3591134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3591959Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3592718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3593479Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3594272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3595139Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3595988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3596812Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3597610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3598474Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3598792Z 2025-12-04T09:41:39.3599098Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3599804Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3600435Z res = mod(**inputs) 2025-12-04T09:41:39.3601689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3602491Z outputs = self.fnet( 2025-12-04T09:41:39.3603233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3604043Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3604830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3605660Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3606405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3607175Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3607978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3608834Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3609689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3615393Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3616216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3617087Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3617406Z 2025-12-04T09:41:39.3617597Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3618325Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3618979Z res = mod(**inputs) 2025-12-04T09:41:39.3619700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3620641Z outputs = self.fnet( 2025-12-04T09:41:39.3621390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3622267Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3623149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3623954Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3624695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3625409Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3626194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3627127Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3628148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3628966Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3629771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3630641Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3630958Z 2025-12-04T09:41:39.3631107Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3631538Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3632275Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3632941Z res = mod(**inputs) 2025-12-04T09:41:39.3633659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3634461Z outputs = self.fnet( 2025-12-04T09:41:39.3635208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3636025Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3636805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3637643Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3638519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3639245Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3640043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3640854Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3641753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3642565Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3643396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3644328Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3645180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3646023Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3646793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3647717Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3648199Z 2025-12-04T09:41:39.3648349Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3648781Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3649205Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3649908Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3650532Z res = mod(**inputs) 2025-12-04T09:41:39.3651301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3652078Z outputs = self.fnet( 2025-12-04T09:41:39.3652780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3653567Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3654329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3655136Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3655857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3656591Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3657380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3658215Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3659028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3659821Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3660609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3661437Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3661757Z 2025-12-04T09:41:39.3661938Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3662645Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3663289Z res = mod(**inputs) 2025-12-04T09:41:39.3663983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3664757Z outputs = self.fnet( 2025-12-04T09:41:39.3665462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3666240Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3667102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3668094Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3668855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3669598Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3670412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3671274Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3672127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3673065Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3673949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3674818Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3675130Z 2025-12-04T09:41:39.3675328Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3676039Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3676698Z res = mod(**inputs) 2025-12-04T09:41:39.3677415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3678274Z outputs = self.fnet( 2025-12-04T09:41:39.3679068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3679884Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3680667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3681484Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3682243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3682999Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3683793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3684655Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3685621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3686418Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3687194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3688037Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3688344Z 2025-12-04T09:41:39.3688537Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3689242Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3689869Z res = mod(**inputs) 2025-12-04T09:41:39.3690576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3691347Z outputs = self.fnet( 2025-12-04T09:41:39.3692054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3692836Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3693601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3694405Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3695135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3695865Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3696647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3697464Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3698279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3699082Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3699863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3700696Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3701385Z 2025-12-04T09:41:39.3701698Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3702201Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3702931Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3703578Z res = mod(**inputs) 2025-12-04T09:41:39.3704307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3705100Z outputs = self.fnet( 2025-12-04T09:41:39.3705826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3706919Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3707713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3708624Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3709370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3710129Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3710937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3711757Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3712607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3713449Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3714314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3715255Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3716152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3717039Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3717826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3718866Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3719359Z 2025-12-04T09:41:39.3719498Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3719880Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3720296Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3721011Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3721661Z res = mod(**inputs) 2025-12-04T09:41:39.3722368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3723127Z outputs = self.fnet( 2025-12-04T09:41:39.3723848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3724625Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3725370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3726172Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3726907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3727635Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3728399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3729223Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3730051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3730843Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3731622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3732469Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3732772Z 2025-12-04T09:41:39.3732966Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3733654Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3734346Z res = mod(**inputs) 2025-12-04T09:41:39.3735052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3735942Z outputs = self.fnet( 2025-12-04T09:41:39.3736963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3737779Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3738565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3739382Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3740141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3740895Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3741766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3742613Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3743470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3744295Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3745108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3745959Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3746286Z 2025-12-04T09:41:39.3746476Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3747289Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3747938Z res = mod(**inputs) 2025-12-04T09:41:39.3748674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3749486Z outputs = self.fnet( 2025-12-04T09:41:39.3750223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3751019Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3751815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3752658Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3753404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3754154Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3754959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3755805Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3756640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3757456Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3758280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3759234Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3759542Z 2025-12-04T09:41:39.3759723Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3760423Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3761062Z res = mod(**inputs) 2025-12-04T09:41:39.3761754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3762525Z outputs = self.fnet( 2025-12-04T09:41:39.3763241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3764102Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3764859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3765713Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3766452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3767182Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3767954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3768790Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3769608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3770388Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3771179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3772020Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3772322Z 2025-12-04T09:41:39.3772472Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3772886Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3773597Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3774234Z res = mod(**inputs) 2025-12-04T09:41:39.3774923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3775697Z outputs = self.fnet( 2025-12-04T09:41:39.3776414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3777207Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3777963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3778778Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3779524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3780263Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3781043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3781857Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3782682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3783490Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3784322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3785256Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3786126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3787226Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3788020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3788985Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3789480Z 2025-12-04T09:41:39.3789633Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3790016Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3790462Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3791198Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3791918Z res = mod(**inputs) 2025-12-04T09:41:39.3792651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3793449Z outputs = self.fnet( 2025-12-04T09:41:39.3794244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3795043Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3795831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3796669Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3797414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3803681Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3804498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3805369Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3806216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3807036Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3807841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3808710Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3809024Z 2025-12-04T09:41:39.3809214Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3809949Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3810617Z res = mod(**inputs) 2025-12-04T09:41:39.3811335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3812142Z outputs = self.fnet( 2025-12-04T09:41:39.3812880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3813800Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3814555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3815371Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3816117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3816842Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3817625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3818464Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3819298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3820087Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3820877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3821718Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3822023Z 2025-12-04T09:41:39.3822218Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3822912Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3823555Z res = mod(**inputs) 2025-12-04T09:41:39.3824269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3825035Z outputs = self.fnet( 2025-12-04T09:41:39.3825916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3826825Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3827885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3828713Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3829473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3830237Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3831038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3831897Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3832750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3833580Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3834380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3835260Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3835596Z 2025-12-04T09:41:39.3835784Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3836517Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3837162Z res = mod(**inputs) 2025-12-04T09:41:39.3837893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3838700Z outputs = self.fnet( 2025-12-04T09:41:39.3839519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3840310Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3841083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3841892Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3842626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3843363Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3844150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3844993Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3845804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3846612Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3847403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3848242Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3848559Z 2025-12-04T09:41:39.3848696Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3849132Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3849844Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3850472Z res = mod(**inputs) 2025-12-04T09:41:39.3851175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3851952Z outputs = self.fnet( 2025-12-04T09:41:39.3852482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3852609Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3853150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3853360Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3853845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3854027Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3854555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3854719Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3855279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3855426Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3856024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3856232Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3856775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3856969Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3857416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3857780Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3857795Z 2025-12-04T09:41:39.3857930Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3858074Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3858261Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3858662Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3858787Z res = mod(**inputs) 2025-12-04T09:41:39.3859326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3859451Z outputs = self.fnet( 2025-12-04T09:41:39.3859982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3860110Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3865630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3865792Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3866275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3866423Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3867075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3867277Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3867830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3867979Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3868538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3868727Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3868743Z 2025-12-04T09:41:39.3868945Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3869358Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3869469Z res = mod(**inputs) 2025-12-04T09:41:39.3870028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3870217Z outputs = self.fnet( 2025-12-04T09:41:39.3870765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3870903Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3871503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3871667Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3872148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3872288Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3872846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3873021Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3873568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3873719Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3874266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3874465Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3874481Z 2025-12-04T09:41:39.3874672Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3875085Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3875208Z res = mod(**inputs) 2025-12-04T09:41:39.3875754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3875883Z outputs = self.fnet( 2025-12-04T09:41:39.3876425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3876556Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3877108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3877260Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3877743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3877890Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3878546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3878730Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3879259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3879393Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3879940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3880119Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3880134Z 2025-12-04T09:41:39.3880333Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3880732Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3880845Z res = mod(**inputs) 2025-12-04T09:41:39.3881392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3881501Z outputs = self.fnet( 2025-12-04T09:41:39.3882029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3882171Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3882700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3882911Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3883384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3883585Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3884132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3884304Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3884839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3884973Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3885498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3885688Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3885707Z 2025-12-04T09:41:39.3885846Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3886030Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3886449Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3886559Z res = mod(**inputs) 2025-12-04T09:41:39.3887105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3887219Z outputs = self.fnet( 2025-12-04T09:41:39.3887752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3887888Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3888421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3888572Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3889056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3889195Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3889736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3889881Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3890437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3890577Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3891170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3891441Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3891973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3892175Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3892635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3892986Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3893000Z 2025-12-04T09:41:39.3893148Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3893279Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3893464Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3893878Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3893984Z res = mod(**inputs) 2025-12-04T09:41:39.3894516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3894701Z outputs = self.fnet( 2025-12-04T09:41:39.3895229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3896897Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3897454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3897603Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3898087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3898224Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3898756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3898935Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3899469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3899616Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3900150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3900336Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3900351Z 2025-12-04T09:41:39.3900547Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3901529Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3901670Z res = mod(**inputs) 2025-12-04T09:41:39.3902223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3902338Z outputs = self.fnet( 2025-12-04T09:41:39.3902899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3903033Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3903580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3903750Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3904232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3904381Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3904929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3905101Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3905661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3905804Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3906351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3906550Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3906570Z 2025-12-04T09:41:39.3906848Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3907275Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3907386Z res = mod(**inputs) 2025-12-04T09:41:39.3907932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3908060Z outputs = self.fnet( 2025-12-04T09:41:39.3908606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3908743Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3909417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3909569Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3910137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3910279Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3910823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3911010Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3911559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3911706Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3912253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3912447Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3912462Z 2025-12-04T09:41:39.3912656Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3913077Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3913201Z res = mod(**inputs) 2025-12-04T09:41:39.3913748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3913864Z outputs = self.fnet( 2025-12-04T09:41:39.3914418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3914548Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3915097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3915258Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3915740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3915887Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3916436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3916615Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3917167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3917306Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3917854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3918052Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3918067Z 2025-12-04T09:41:39.3918213Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3918520Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3918921Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3919030Z res = mod(**inputs) 2025-12-04T09:41:39.3919579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3919692Z outputs = self.fnet( 2025-12-04T09:41:39.3920235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3920358Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3920898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3921063Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3921535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3921731Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3922334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3922483Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3923176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3923315Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3924178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3924414Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3924964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3925193Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3925657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3926028Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3926043Z 2025-12-04T09:41:39.3926209Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3926354Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3926548Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3926985Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3927101Z res = mod(**inputs) 2025-12-04T09:41:39.3927673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3927795Z outputs = self.fnet( 2025-12-04T09:41:39.3928354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3928504Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3929069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3929225Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3929723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3929866Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3930430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3930612Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3931160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3931326Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3931878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3932092Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3932107Z 2025-12-04T09:41:39.3932301Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3932718Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3932852Z res = mod(**inputs) 2025-12-04T09:41:39.3933402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3933526Z outputs = self.fnet( 2025-12-04T09:41:39.3934097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3934291Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3934861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3935016Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3935686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3935844Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3936379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3936573Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3937104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3937244Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3937796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3937984Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3937999Z 2025-12-04T09:41:39.3938185Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3938603Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3938713Z res = mod(**inputs) 2025-12-04T09:41:39.3939260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3939375Z outputs = self.fnet( 2025-12-04T09:41:39.3939906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3940048Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3940580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3940746Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3941216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3941359Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3941902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3942076Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3942608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3942761Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3943296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3943497Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3943516Z 2025-12-04T09:41:39.3943703Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3944107Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3944233Z res = mod(**inputs) 2025-12-04T09:41:39.3944770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3944903Z outputs = self.fnet( 2025-12-04T09:41:39.3945436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3945567Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3946114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3946263Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3946819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3947197Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3947885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 248, in forward 2025-12-04T09:41:39.3948084Z self_fourier_outputs = self.fourier(hidden_states) 2025-12-04T09:41:39.3948632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 201, in forward 2025-12-04T09:41:39.3948774Z self_outputs = self.self(hidden_states) 2025-12-04T09:41:39.3949343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 180, in forward 2025-12-04T09:41:39.3949536Z outputs = self.fourier_transform(hidden_states).real 2025-12-04T09:41:39.3949551Z 2025-12-04T09:41:39.3949712Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3949906Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3950333Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3950463Z res = mod(**inputs) 2025-12-04T09:41:39.3951017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 670, in forward 2025-12-04T09:41:39.3951138Z outputs = self.fnet( 2025-12-04T09:41:39.3951701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 517, in forward 2025-12-04T09:41:39.3951831Z encoder_outputs = self.encoder( 2025-12-04T09:41:39.3952398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 279, in forward 2025-12-04T09:41:39.3952553Z layer_outputs = layer_module(hidden_states) 2025-12-04T09:41:39.3953040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:41:39.3953198Z return super().__call__(*args, **kwargs) 2025-12-04T09:41:39.3953745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 251, in forward 2025-12-04T09:41:39.3953896Z layer_output = apply_chunking_to_forward( 2025-12-04T09:41:39.3954495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:41:39.3954633Z return forward_fn(*input_tensors) 2025-12-04T09:41:39.3955267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 260, in feed_forward_chunk 2025-12-04T09:41:39.3955480Z intermediate_output = self.intermediate(fourier_output) 2025-12-04T09:41:39.3956030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 219, in forward 2025-12-04T09:41:39.3956248Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:41:39.3956717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 62, in forward 2025-12-04T09:41:39.3957096Z return 0.5 * input * (1.0 + torch.tanh(math.sqrt(2.0 / math.pi) * (input + 0.044715 * torch.pow(input, 3.0)))) 2025-12-04T09:41:39.3957111Z 2025-12-04T09:41:39.3957258Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3957398Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3957552Z cudagraph partition due to non gpu ops 2025-12-04T09:41:39.3957746Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:41:39.3958164Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:41:39.3958292Z res = mod(**inputs) 2025-12-04T09:41:39.3958849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/fnet/modeling_fnet.py", line 685, in forward 2025-12-04T09:41:39.3959335Z masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:41:39.3959403Z 2025-12-04T09:41:45.6505959Z Compilation time (from dynamo_timed): 15.869685822 2025-12-04T09:41:45.6596517Z pass 2025-12-04T09:41:45.6597122Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:45.6599361Z TIMING: _recursive_pre_grad_passes:0.03273 _recursive_joint_graph_passes:0.35542 _recursive_post_grad_passes:0.0928 async_compile.wait:1.10444 code_gen:5.88411 inductor_compile:9.43203 backend_compile:12.98081 gc:0.00118 entire_frame_compile:15.86969 total_wall_time:15.86969 2025-12-04T09:41:45.6602122Z STATS: call_* op count: 232 | FakeTensorMode.__torch_dispatch__:11067 | FakeTensor.__torch_dispatch__:2950 | ProxyTorchDispatchMode.__torch_dispatch__:1323 2025-12-04T09:41:45.6603271Z Dynamo produced 1 graphs covering 232 ops with 0 graph breaks (0 unique) 2025-12-04T09:41:48.2895955Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:41:48.2897721Z import pynvml # type: ignore[import] 2025-12-04T09:41:53.2870298Z 2025-12-04T09:41:55.4622755Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:41:55.4623378Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:41:55.4644438Z cpu eval LayoutLMForMaskedLM 2025-12-04T09:41:56.4882995Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:56.8544637Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:41:57.2383983Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:13.7815890Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7816381Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7816879Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7817262Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7817790Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7818239Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7818686Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7819060Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7819554Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7820079Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7820596Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.7821468Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.7822386Z res = mod(**inputs) 2025-12-04T09:42:13.7823098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7823983Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7824967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.7825990Z outputs = self.layoutlm( 2025-12-04T09:42:13.7826903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7827971Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7828982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.7830081Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.7831053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7832077Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7833077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.7834476Z layer_outputs = layer_module( 2025-12-04T09:42:13.7835394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.7836318Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.7837505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.7838749Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.7840121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.7841257Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.7853708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.7854798Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.7855779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.7856758Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.7857549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.7858250Z return self.act(input) 2025-12-04T09:42:13.7858525Z 2025-12-04T09:42:13.7858672Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7859117Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7859561Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7859983Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7860413Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7860853Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7861283Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7861668Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7862111Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7862530Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7862915Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7863353Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.7864070Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.7864731Z res = mod(**inputs) 2025-12-04T09:42:13.7865383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7866112Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7867020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.7873101Z outputs = self.layoutlm( 2025-12-04T09:42:13.7873813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7874559Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7875417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.7876283Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.7877007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7877743Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7878597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.7879458Z layer_outputs = layer_module( 2025-12-04T09:42:13.7880180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.7880938Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.7881808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.7884166Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.7885190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.7886014Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.7886903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.7887901Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.7888821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.7889739Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.7890511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.7891220Z return self.act(input) 2025-12-04T09:42:13.7891423Z 2025-12-04T09:42:13.7891561Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7891950Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7892338Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7892705Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7893084Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7893464Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7893828Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7894209Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7894591Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7894969Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7895336Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7895774Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.7896496Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.7897142Z res = mod(**inputs) 2025-12-04T09:42:13.7897791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7898529Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7899348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.7900189Z outputs = self.layoutlm( 2025-12-04T09:42:13.7901475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7902248Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7903097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.7903976Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.7904704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7905460Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7906305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.7907282Z layer_outputs = layer_module( 2025-12-04T09:42:13.7908005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.7908762Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.7909630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.7910525Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.7911378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.7912365Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.7913282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.7914423Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.7915384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.7916317Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.7917117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.7917841Z return self.act(input) 2025-12-04T09:42:13.7918050Z 2025-12-04T09:42:13.7918195Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7918704Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7919089Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7919464Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7919846Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7920226Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7920606Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7920974Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7921354Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7921734Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7922102Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7922535Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.7923259Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.7923896Z res = mod(**inputs) 2025-12-04T09:42:13.7924539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7925265Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7926101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.7926931Z outputs = self.layoutlm( 2025-12-04T09:42:13.7927606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7928327Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7929147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.7929992Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.7930837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7931839Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7932685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.7933560Z layer_outputs = layer_module( 2025-12-04T09:42:13.7934281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.7935051Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.7935920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.7936812Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.7937666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.7938497Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.7939412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.7940442Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.7941492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.7942416Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.7943381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.7944083Z return self.act(input) 2025-12-04T09:42:13.7944288Z 2025-12-04T09:42:13.7944441Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7944812Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7945195Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7945573Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7945939Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7946315Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7946778Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7947422Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7947823Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7948214Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7948607Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7949042Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.7949789Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.7950467Z res = mod(**inputs) 2025-12-04T09:42:13.7951119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7951869Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7952731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.7953601Z outputs = self.layoutlm( 2025-12-04T09:42:13.7954281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7955034Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7955893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.7956747Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.7957467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7958223Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7959168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.7959995Z layer_outputs = layer_module( 2025-12-04T09:42:13.7960696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.7961440Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.7962269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.7963138Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.7963965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.7964784Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.7965660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.7966657Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.7967584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.7968499Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.7969258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.7970042Z return self.act(input) 2025-12-04T09:42:13.7970242Z 2025-12-04T09:42:13.7970398Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7970775Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7971225Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7971610Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7971986Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7972354Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7972718Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7973102Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7973476Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7973842Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7974218Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7974649Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.7975363Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.7976011Z res = mod(**inputs) 2025-12-04T09:42:13.7976670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7977403Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7978222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.7979060Z outputs = self.layoutlm( 2025-12-04T09:42:13.7979737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7980457Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7981272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.7982117Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.7982818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.7983541Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.7984355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.7985188Z layer_outputs = layer_module( 2025-12-04T09:42:13.7985894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.7986621Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.7987741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.7988633Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.7989493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.7990336Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.7991252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.7992288Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.7993366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.7994383Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.7995189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.7995914Z return self.act(input) 2025-12-04T09:42:13.7996124Z 2025-12-04T09:42:13.7996267Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7996673Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7997070Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7997540Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7997937Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7998331Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7998787Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7999235Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.7999632Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8000024Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8000403Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8001305Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.8002055Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.8002716Z res = mod(**inputs) 2025-12-04T09:42:13.8003384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8004135Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8004995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.8005841Z outputs = self.layoutlm( 2025-12-04T09:42:13.8006537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8007279Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8008118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.8008981Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.8009700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8010442Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8011280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.8012149Z layer_outputs = layer_module( 2025-12-04T09:42:13.8012869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.8013727Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.8014558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.8015410Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.8016236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.8017039Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.8017924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.8018914Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.8019846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.8020746Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.8021524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.8022226Z return self.act(input) 2025-12-04T09:42:13.8022428Z 2025-12-04T09:42:13.8022578Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8022951Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8023328Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8023704Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8024066Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8024441Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8024816Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8025179Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8025698Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8026076Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8026440Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8026978Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.8027993Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.8028670Z res = mod(**inputs) 2025-12-04T09:42:13.8029323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8030076Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8030934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.8031783Z outputs = self.layoutlm( 2025-12-04T09:42:13.8032474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8033228Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8034078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.8034938Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.8035657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8036393Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8037243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.8038091Z layer_outputs = layer_module( 2025-12-04T09:42:13.8038804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.8039658Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.8040488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.8041349Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.8042177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.8042993Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.8043868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.8044862Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.8045791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.8046697Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.8047456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.8048158Z return self.act(input) 2025-12-04T09:42:13.8048356Z 2025-12-04T09:42:13.8048505Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8048878Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8049258Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8049640Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8050006Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8050383Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8050760Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8051134Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8051500Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8051880Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8052257Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8052674Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.8053394Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.8054123Z res = mod(**inputs) 2025-12-04T09:42:13.8054757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8060496Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8061512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.8062384Z outputs = self.layoutlm( 2025-12-04T09:42:13.8063070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8063821Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8064687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.8065547Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.8066271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8067131Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8067995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.8068845Z layer_outputs = layer_module( 2025-12-04T09:42:13.8069568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.8070339Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.8071217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.8072097Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.8072947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.8073805Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.8074700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.8075716Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.8076675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.8077604Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.8078385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.8079190Z return self.act(input) 2025-12-04T09:42:13.8079385Z 2025-12-04T09:42:13.8079524Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8079900Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8080263Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8080629Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8080994Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8081354Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8081719Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8082083Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8082445Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8082805Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8083167Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8083581Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.8084290Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.8084929Z res = mod(**inputs) 2025-12-04T09:42:13.8085567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8086275Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8087095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.8088006Z outputs = self.layoutlm( 2025-12-04T09:42:13.8088720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8089428Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8090256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.8091087Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.8091764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8092469Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8093288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.8094120Z layer_outputs = layer_module( 2025-12-04T09:42:13.8094800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.8095531Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.8096365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.8097210Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.8098020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.8098828Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.8099721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.8100701Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.8102215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.8103166Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.8103968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.8104684Z return self.act(input) 2025-12-04T09:42:13.8104903Z 2025-12-04T09:42:13.8105048Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8105446Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8105825Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8106213Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8106599Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8107054Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8107442Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8107827Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8108216Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8108593Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8108984Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8109428Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.8110156Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.8110822Z res = mod(**inputs) 2025-12-04T09:42:13.8111486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8112219Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8113076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.8113932Z outputs = self.layoutlm( 2025-12-04T09:42:13.8114624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8115500Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8116356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.8117228Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.8122919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8123690Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8124593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.8125461Z layer_outputs = layer_module( 2025-12-04T09:42:13.8126171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.8126942Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.8127809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.8128703Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.8129546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.8130392Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.8131312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.8132335Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.8133292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.8134376Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.8135150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.8135847Z return self.act(input) 2025-12-04T09:42:13.8136065Z 2025-12-04T09:42:13.8136205Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8136592Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8136974Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8137347Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8137733Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8138113Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8138481Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8138865Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8139248Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8139612Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8139987Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8140415Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.8141120Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.8141774Z res = mod(**inputs) 2025-12-04T09:42:13.8142417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8143134Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8143954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 721, in forward 2025-12-04T09:42:13.8144793Z outputs = self.layoutlm( 2025-12-04T09:42:13.8145460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8146169Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8147257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 613, in forward 2025-12-04T09:42:13.8148124Z encoder_outputs = self.encoder( 2025-12-04T09:42:13.8148842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8149676Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8150596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 369, in forward 2025-12-04T09:42:13.8151462Z layer_outputs = layer_module( 2025-12-04T09:42:13.8152179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:42:13.8152933Z return super().__call__(*args, **kwargs) 2025-12-04T09:42:13.8153795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 328, in forward 2025-12-04T09:42:13.8154683Z layer_output = apply_chunking_to_forward( 2025-12-04T09:42:13.8155516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:42:13.8156360Z return forward_fn(*input_tensors) 2025-12-04T09:42:13.8157268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 336, in feed_forward_chunk 2025-12-04T09:42:13.8158298Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:42:13.8159356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 281, in forward 2025-12-04T09:42:13.8160268Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:42:13.8161040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:42:13.8161737Z return self.act(input) 2025-12-04T09:42:13.8161938Z 2025-12-04T09:42:13.8162076Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8162460Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8162838Z cudagraph partition due to non gpu ops 2025-12-04T09:42:13.8163254Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:42:13.8163970Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:42:13.8164616Z res = mod(**inputs) 2025-12-04T09:42:13.8165254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:42:13.8165979Z output = func(self, *args, **kwargs) 2025-12-04T09:42:13.8166803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/layoutlm/modeling_layoutlm.py", line 740, in forward 2025-12-04T09:42:13.8167635Z masked_lm_loss = loss_fct( 2025-12-04T09:42:13.8167845Z 2025-12-04T09:42:20.1432478Z Compilation time (from dynamo_timed): 21.860468967 2025-12-04T09:42:20.1526122Z pass 2025-12-04T09:42:20.1526741Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:20.1528411Z TIMING: _recursive_pre_grad_passes:0.05626 _recursive_joint_graph_passes:0.74863 _recursive_post_grad_passes:0.09261 async_compile.wait:0.90398 code_gen:5.62384 inductor_compile:10.02927 backend_compile:17.29911 gc:0.00047 entire_frame_compile:21.86047 total_wall_time:21.86047 2025-12-04T09:42:20.1530362Z STATS: call_* op count: 432 | FakeTensorMode.__torch_dispatch__:20793 | FakeTensor.__torch_dispatch__:3620 | ProxyTorchDispatchMode.__torch_dispatch__:3441 2025-12-04T09:42:20.1531393Z Dynamo produced 1 graphs covering 432 ops with 0 graph breaks (0 unique) 2025-12-04T09:42:23.1571865Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:42:23.1573480Z import pynvml # type: ignore[import] 2025-12-04T09:42:28.1648618Z 2025-12-04T09:42:40.7591829Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:42:40.7592268Z loading model: 0it [00:12, ?it/s] 2025-12-04T09:42:40.7633410Z cpu eval M2M100ForConditionalGeneration 2025-12-04T09:42:42.0896397Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:42.6237318Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:42:43.1592355Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:15.3542431Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3544756Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3545530Z res = mod(**inputs) 2025-12-04T09:43:15.3546555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3547943Z outputs = self.model( 2025-12-04T09:43:15.3549081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3550217Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3551237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 849, in forward 2025-12-04T09:43:15.3552447Z embed_pos = self.embed_positions(input_ids, inputs_embeds) 2025-12-04T09:43:15.3553613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context 2025-12-04T09:43:15.3554483Z return func(*args, **kwargs) 2025-12-04T09:43:15.3555710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 149, in forward 2025-12-04T09:43:15.3557484Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length).to( 2025-12-04T09:43:15.3559280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 81, in create_position_ids_from_input_ids 2025-12-04T09:43:15.3560733Z mask = input_ids.ne(padding_idx).int() 2025-12-04T09:43:15.3561121Z 2025-12-04T09:43:15.3561338Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3561940Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3562471Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3562955Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3563433Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3563958Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3564405Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3564847Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3565351Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3565787Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3566225Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3566682Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3567203Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3567983Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3568760Z res = mod(**inputs) 2025-12-04T09:43:15.3569693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3570656Z outputs = self.model( 2025-12-04T09:43:15.3571532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3572530Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3573416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 849, in forward 2025-12-04T09:43:15.3574501Z embed_pos = self.embed_positions(input_ids, inputs_embeds) 2025-12-04T09:43:15.3575512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context 2025-12-04T09:43:15.3576716Z return func(*args, **kwargs) 2025-12-04T09:43:15.3577610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 149, in forward 2025-12-04T09:43:15.3579062Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length).to( 2025-12-04T09:43:15.3580587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 82, in create_position_ids_from_input_ids 2025-12-04T09:43:15.3581930Z incremental_indices = (torch.cumsum(mask, dim=1).type_as(mask) + past_key_values_length) * mask 2025-12-04T09:43:15.3587832Z 2025-12-04T09:43:15.3588107Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3589017Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3589867Z res = mod(**inputs) 2025-12-04T09:43:15.3590782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3591790Z outputs = self.model( 2025-12-04T09:43:15.3592697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3593717Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3594674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 849, in forward 2025-12-04T09:43:15.3595758Z embed_pos = self.embed_positions(input_ids, inputs_embeds) 2025-12-04T09:43:15.3596760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context 2025-12-04T09:43:15.3597639Z return func(*args, **kwargs) 2025-12-04T09:43:15.3598625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 149, in forward 2025-12-04T09:43:15.3599929Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length).to( 2025-12-04T09:43:15.3601813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 82, in create_position_ids_from_input_ids 2025-12-04T09:43:15.3603462Z incremental_indices = (torch.cumsum(mask, dim=1).type_as(mask) + past_key_values_length) * mask 2025-12-04T09:43:15.3604148Z 2025-12-04T09:43:15.3604364Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3604824Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3605298Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3605729Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3606259Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3606723Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3607161Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3607682Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3608552Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3609278Z res = mod(**inputs) 2025-12-04T09:43:15.3610174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3611209Z outputs = self.model( 2025-12-04T09:43:15.3612111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3613162Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3614087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3615019Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3615851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3616916Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3617854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3618929Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3619850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3620809Z return func(*args, **kwargs) 2025-12-04T09:43:15.3621715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3622690Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3623686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3624868Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3625243Z 2025-12-04T09:43:15.3625453Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3625892Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3626380Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3627642Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3628518Z res = mod(**inputs) 2025-12-04T09:43:15.3629344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3630378Z outputs = self.model( 2025-12-04T09:43:15.3631294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3632305Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3633264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3634279Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3635201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3636097Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3637078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3638138Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3638554Z 2025-12-04T09:43:15.3638745Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3639367Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3639858Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3640365Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3640837Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3641312Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3641762Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3642132Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3642606Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3643422Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3644237Z res = mod(**inputs) 2025-12-04T09:43:15.3645517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3646365Z outputs = self.model( 2025-12-04T09:43:15.3647155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3647989Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3648811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3649860Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3650722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3651596Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3652843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3654035Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3655092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3656046Z return func(*args, **kwargs) 2025-12-04T09:43:15.3657264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3658448Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3659664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3660912Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3661393Z 2025-12-04T09:43:15.3661638Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3662133Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3662737Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3663681Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3664339Z res = mod(**inputs) 2025-12-04T09:43:15.3665233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3666190Z outputs = self.model( 2025-12-04T09:43:15.3667297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3668312Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3669184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3670166Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3671055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3672069Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3673105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3674448Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3674912Z 2025-12-04T09:43:15.3675125Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3675699Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3676238Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3676749Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3677250Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3677747Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3678241Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3678834Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3679487Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3680394Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3681158Z res = mod(**inputs) 2025-12-04T09:43:15.3682151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3683172Z outputs = self.model( 2025-12-04T09:43:15.3684112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3685042Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3686068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3687005Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3687917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3688790Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3689733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3690774Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3691715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3692645Z return func(*args, **kwargs) 2025-12-04T09:43:15.3693591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3694574Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3695623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3696735Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3697190Z 2025-12-04T09:43:15.3697360Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3698116Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3698629Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3699447Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3700221Z res = mod(**inputs) 2025-12-04T09:43:15.3701729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3702755Z outputs = self.model( 2025-12-04T09:43:15.3703868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3704881Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3705818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3706702Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3707767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3708541Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3709360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3710279Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3710648Z 2025-12-04T09:43:15.3710795Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3711206Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3711586Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3711980Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3712373Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3712762Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3713163Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3713559Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3713992Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3714735Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3715402Z res = mod(**inputs) 2025-12-04T09:43:15.3716165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3716986Z outputs = self.model( 2025-12-04T09:43:15.3717765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3718772Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3719769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3733066Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3734012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3734791Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3735605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3736461Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3737284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3738071Z return func(*args, **kwargs) 2025-12-04T09:43:15.3738854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3739722Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3740664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3741654Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3742046Z 2025-12-04T09:43:15.3742197Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3742593Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3743038Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3743754Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3744413Z res = mod(**inputs) 2025-12-04T09:43:15.3745165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3745976Z outputs = self.model( 2025-12-04T09:43:15.3746856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3747890Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3748714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3749545Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3750283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3751061Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3751886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3752816Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3753186Z 2025-12-04T09:43:15.3753332Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3753738Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3754128Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3754524Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3754915Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3755297Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3755691Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3756085Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3756521Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3757272Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3757949Z res = mod(**inputs) 2025-12-04T09:43:15.3758717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3759757Z outputs = self.model( 2025-12-04T09:43:15.3760517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3761411Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3762192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3763005Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3763713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3764466Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3765266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3766122Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3766944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3767725Z return func(*args, **kwargs) 2025-12-04T09:43:15.3768492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3769349Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3775844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3776873Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3777270Z 2025-12-04T09:43:15.3777418Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3777827Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3778284Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3779013Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3779702Z res = mod(**inputs) 2025-12-04T09:43:15.3780477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3781305Z outputs = self.model( 2025-12-04T09:43:15.3782084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3782932Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3783743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3784684Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3785394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3786146Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3787099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3788194Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3788565Z 2025-12-04T09:43:15.3788718Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3789128Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3789513Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3789913Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3790308Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3790688Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3791080Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3791473Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3791923Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3792646Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3793418Z res = mod(**inputs) 2025-12-04T09:43:15.3794184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3795010Z outputs = self.model( 2025-12-04T09:43:15.3795859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3796710Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3797526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3798346Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3799174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3799924Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3800721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3802166Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3803012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3803824Z return func(*args, **kwargs) 2025-12-04T09:43:15.3804616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3805503Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3806462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3807504Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3807882Z 2025-12-04T09:43:15.3808032Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3808437Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3808897Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3809625Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3810303Z res = mod(**inputs) 2025-12-04T09:43:15.3811081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3811922Z outputs = self.model( 2025-12-04T09:43:15.3812685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3813640Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3814434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3815235Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3815948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3816698Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3817522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3818414Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3818776Z 2025-12-04T09:43:15.3818919Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3819316Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3819691Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3820083Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3820470Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3820861Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3821231Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3821616Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3822054Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3822902Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3823563Z res = mod(**inputs) 2025-12-04T09:43:15.3824393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3825206Z outputs = self.model( 2025-12-04T09:43:15.3825951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3826858Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3827844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3828663Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3829394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3830172Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3831013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3831878Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3837478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3838290Z return func(*args, **kwargs) 2025-12-04T09:43:15.3839081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3839969Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3840935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3841973Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3842362Z 2025-12-04T09:43:15.3842509Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3842915Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3843369Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3844124Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3844785Z res = mod(**inputs) 2025-12-04T09:43:15.3845556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3846393Z outputs = self.model( 2025-12-04T09:43:15.3847161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3848124Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3848916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3849737Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3850433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3851183Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3851999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3852889Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3853246Z 2025-12-04T09:43:15.3853388Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3853784Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3854173Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3854545Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3854931Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3855317Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3855781Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3856168Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3856608Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3857395Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3858054Z res = mod(**inputs) 2025-12-04T09:43:15.3858805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3859625Z outputs = self.model( 2025-12-04T09:43:15.3860365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3861186Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3861977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3862784Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3863497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3864228Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3865040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3865892Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3866701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3867754Z return func(*args, **kwargs) 2025-12-04T09:43:15.3868573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3869461Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3870421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3871448Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3871845Z 2025-12-04T09:43:15.3871990Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3872402Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3872839Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3873579Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3874251Z res = mod(**inputs) 2025-12-04T09:43:15.3875017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3875838Z outputs = self.model( 2025-12-04T09:43:15.3876624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3877479Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3878261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3879243Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3879938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3880670Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3881459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3882341Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3882674Z 2025-12-04T09:43:15.3882814Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3883175Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3883545Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3883906Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3884348Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3884710Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3885074Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3885437Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3885906Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3886609Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3887245Z res = mod(**inputs) 2025-12-04T09:43:15.3887968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3888767Z outputs = self.model( 2025-12-04T09:43:15.3889497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3890300Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3891067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3891866Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3892563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3893286Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3894070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3895022Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3896119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3896896Z return func(*args, **kwargs) 2025-12-04T09:43:15.3897672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3898542Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3899476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3900556Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3901339Z 2025-12-04T09:43:15.3901483Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3901877Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3902306Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3903038Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3903700Z res = mod(**inputs) 2025-12-04T09:43:15.3904454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3905266Z outputs = self.model( 2025-12-04T09:43:15.3906034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3906965Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3907767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3908580Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3909290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3910044Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3910861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3911770Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3912131Z 2025-12-04T09:43:15.3912269Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3912796Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3913166Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3913552Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3913935Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3914305Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3915920Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3916306Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3916732Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3917456Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3918110Z res = mod(**inputs) 2025-12-04T09:43:15.3918957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3919745Z outputs = self.model( 2025-12-04T09:43:15.3920484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3921287Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3922049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3922851Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3923542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3924268Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3925048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3925878Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3926661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3927411Z return func(*args, **kwargs) 2025-12-04T09:43:15.3928154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3928993Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3929910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3930883Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3931253Z 2025-12-04T09:43:15.3931387Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3931759Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3932183Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3932877Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3933512Z res = mod(**inputs) 2025-12-04T09:43:15.3934244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3935027Z outputs = self.model( 2025-12-04T09:43:15.3935762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3936565Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3937333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3938121Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3938806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3939531Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3940323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3941193Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3941635Z 2025-12-04T09:43:15.3941768Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3942140Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3942502Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3942925Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3943293Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3943646Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3944012Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3944378Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3944793Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3945485Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3946127Z res = mod(**inputs) 2025-12-04T09:43:15.3946960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3947991Z outputs = self.model( 2025-12-04T09:43:15.3948755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3949592Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3950394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3951213Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3951922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3952671Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3953478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3954334Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3955154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3955943Z return func(*args, **kwargs) 2025-12-04T09:43:15.3956717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3957753Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3958699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3959713Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3960083Z 2025-12-04T09:43:15.3960220Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3960602Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3961038Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3961748Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3962408Z res = mod(**inputs) 2025-12-04T09:43:15.3963155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3963969Z outputs = self.model( 2025-12-04T09:43:15.3964722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3965548Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3966343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3967151Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3967868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3968620Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3969548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3970523Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.3970870Z 2025-12-04T09:43:15.3971006Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3971439Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3971801Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3972170Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3972537Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3972904Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3973264Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3973634Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3974065Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3974773Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3975423Z res = mod(**inputs) 2025-12-04T09:43:15.3976171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3976970Z outputs = self.model( 2025-12-04T09:43:15.3977730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3978551Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3979342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3980141Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3980848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3981595Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3982401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 381, in forward 2025-12-04T09:43:15.3983232Z hidden_states, attn_weights = self.self_attn( 2025-12-04T09:43:15.3984042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.3984829Z return func(*args, **kwargs) 2025-12-04T09:43:15.3985588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.3986439Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.3987630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.3988665Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.3989044Z 2025-12-04T09:43:15.3989191Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3989593Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.3990055Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.3990785Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.3991456Z res = mod(**inputs) 2025-12-04T09:43:15.3992227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.3993064Z outputs = self.model( 2025-12-04T09:43:15.3993828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1258, in forward 2025-12-04T09:43:15.3994669Z encoder_outputs = self.encoder( 2025-12-04T09:43:15.3995484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 883, in forward 2025-12-04T09:43:15.3996317Z layer_outputs = encoder_layer( 2025-12-04T09:43:15.3997032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.3997873Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.3998707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 392, in forward 2025-12-04T09:43:15.3999772Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4000129Z 2025-12-04T09:43:15.4000268Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4000660Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4001626Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4002014Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4002414Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4002812Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4003191Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4003586Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4004037Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4004770Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4005439Z res = mod(**inputs) 2025-12-04T09:43:15.4006214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4007047Z outputs = self.model( 2025-12-04T09:43:15.4007817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4008659Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4009485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4010311Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4011040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4011811Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4012624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4013404Z return func(*args, **kwargs) 2025-12-04T09:43:15.4014296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4015164Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4015994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4016760Z return func(*args, **kwargs) 2025-12-04T09:43:15.4017540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4018400Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4019309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4025594Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4025993Z 2025-12-04T09:43:15.4026143Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4026556Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4027094Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4027495Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4027893Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4028274Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4028673Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4029068Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4029498Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4030242Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4030911Z res = mod(**inputs) 2025-12-04T09:43:15.4031825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4032649Z outputs = self.model( 2025-12-04T09:43:15.4033498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4034338Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4035142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4035984Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4036709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4037479Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4038271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4039189Z return func(*args, **kwargs) 2025-12-04T09:43:15.4039960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4040837Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4041664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4042437Z return func(*args, **kwargs) 2025-12-04T09:43:15.4043206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4044048Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4044972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4045972Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4046347Z 2025-12-04T09:43:15.4046501Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4046877Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4047321Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4048046Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4048686Z res = mod(**inputs) 2025-12-04T09:43:15.4049429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4050240Z outputs = self.model( 2025-12-04T09:43:15.4050996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4051799Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4052595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4053417Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4054126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4054860Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4055640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4056417Z return func(*args, **kwargs) 2025-12-04T09:43:15.4057171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4058069Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4058422Z 2025-12-04T09:43:15.4058560Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4058949Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4059320Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4059765Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4060152Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4060520Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4060905Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4061343Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4061767Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4062487Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4063142Z res = mod(**inputs) 2025-12-04T09:43:15.4063880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4064673Z outputs = self.model( 2025-12-04T09:43:15.4065425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4066251Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4067332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4068179Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4068913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4069683Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4070477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4071278Z return func(*args, **kwargs) 2025-12-04T09:43:15.4072076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4072964Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4073797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4074603Z return func(*args, **kwargs) 2025-12-04T09:43:15.4075397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4076271Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4077226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4078252Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4078630Z 2025-12-04T09:43:15.4078788Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4079178Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4079675Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4080057Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4080424Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4080806Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4081193Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4081560Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4082001Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4088105Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4088789Z res = mod(**inputs) 2025-12-04T09:43:15.4089546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4090390Z outputs = self.model( 2025-12-04T09:43:15.4091170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4092018Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4092828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4093786Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4094522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4095337Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4096143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4096934Z return func(*args, **kwargs) 2025-12-04T09:43:15.4097716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4098719Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4099542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4100299Z return func(*args, **kwargs) 2025-12-04T09:43:15.4101585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4102473Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4103418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4104431Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4104804Z 2025-12-04T09:43:15.4104947Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4105326Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4105755Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4106483Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4107239Z res = mod(**inputs) 2025-12-04T09:43:15.4107986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4108813Z outputs = self.model( 2025-12-04T09:43:15.4109564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4110395Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4111204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4112030Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4112732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4113485Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4114277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4115052Z return func(*args, **kwargs) 2025-12-04T09:43:15.4115830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4116746Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4117100Z 2025-12-04T09:43:15.4117251Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4117635Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4118022Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4118513Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4118873Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4119238Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4119605Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4119968Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4120385Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4121092Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4121874Z res = mod(**inputs) 2025-12-04T09:43:15.4122590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4123392Z outputs = self.model( 2025-12-04T09:43:15.4124208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4125008Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4125786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4126581Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4127276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4128000Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4128760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4129524Z return func(*args, **kwargs) 2025-12-04T09:43:15.4130278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4131130Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4131935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4132694Z return func(*args, **kwargs) 2025-12-04T09:43:15.4133442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4134285Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4135188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4136170Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4136535Z 2025-12-04T09:43:15.4136672Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4137048Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4137419Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4137782Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4138155Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4138527Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4138888Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4139242Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4139662Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4140364Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4140987Z res = mod(**inputs) 2025-12-04T09:43:15.4141715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4142523Z outputs = self.model( 2025-12-04T09:43:15.4143256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4144062Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4144930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4146055Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4146826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4147588Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4148383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4149175Z return func(*args, **kwargs) 2025-12-04T09:43:15.4150002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4151006Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4151939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4152732Z return func(*args, **kwargs) 2025-12-04T09:43:15.4153531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4154411Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4155364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4156379Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4156775Z 2025-12-04T09:43:15.4156921Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4157335Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4157793Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4158633Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4159290Z res = mod(**inputs) 2025-12-04T09:43:15.4160041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4160842Z outputs = self.model( 2025-12-04T09:43:15.4161596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4162424Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4163227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4164025Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4164743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4165498Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4166270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4167055Z return func(*args, **kwargs) 2025-12-04T09:43:15.4167830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4168724Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4169065Z 2025-12-04T09:43:15.4169204Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4169592Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4169974Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4170343Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4170727Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4171108Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4171488Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4171855Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4172288Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4173008Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4173644Z res = mod(**inputs) 2025-12-04T09:43:15.4174384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4175202Z outputs = self.model( 2025-12-04T09:43:15.4175946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4176767Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4177555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4178438Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4179128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4179946Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4180728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4181513Z return func(*args, **kwargs) 2025-12-04T09:43:15.4182273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4183129Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4183950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4184715Z return func(*args, **kwargs) 2025-12-04T09:43:15.4185492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4186347Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4187573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4188597Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4188995Z 2025-12-04T09:43:15.4189142Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4189548Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4189931Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4190327Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4190721Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4191115Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4191493Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4191892Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4192340Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4193072Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4193740Z res = mod(**inputs) 2025-12-04T09:43:15.4194514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4195349Z outputs = self.model( 2025-12-04T09:43:15.4196109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4196954Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4197770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4198594Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4199426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4200174Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4201547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4202347Z return func(*args, **kwargs) 2025-12-04T09:43:15.4203147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4204056Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4204906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4205706Z return func(*args, **kwargs) 2025-12-04T09:43:15.4206505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4207692Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4208640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4209757Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4210155Z 2025-12-04T09:43:15.4210302Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4210706Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4211147Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4211891Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4212565Z res = mod(**inputs) 2025-12-04T09:43:15.4213317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4214161Z outputs = self.model( 2025-12-04T09:43:15.4214939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4215793Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4216605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4217443Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4218172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4218929Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4219832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4220605Z return func(*args, **kwargs) 2025-12-04T09:43:15.4221377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4222262Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4222621Z 2025-12-04T09:43:15.4222764Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4223160Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4223533Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4223925Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4224313Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4224701Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4225068Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4225454Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4225892Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4226599Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4227513Z res = mod(**inputs) 2025-12-04T09:43:15.4228291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4229129Z outputs = self.model( 2025-12-04T09:43:15.4229914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4230771Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4231594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4232425Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4233157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4233927Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4234734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4235577Z return func(*args, **kwargs) 2025-12-04T09:43:15.4236379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4237340Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4238288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4239263Z return func(*args, **kwargs) 2025-12-04T09:43:15.4240037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4240896Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4241809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4242807Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4243173Z 2025-12-04T09:43:15.4243325Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4243725Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4244099Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4244484Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4244868Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4245241Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4245624Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4246007Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4246429Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4247152Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4247801Z res = mod(**inputs) 2025-12-04T09:43:15.4248544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4249343Z outputs = self.model( 2025-12-04T09:43:15.4258438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4259306Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4260107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4260926Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4261633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4262370Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4263136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4263902Z return func(*args, **kwargs) 2025-12-04T09:43:15.4264665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4265523Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4266359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4267422Z return func(*args, **kwargs) 2025-12-04T09:43:15.4268223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4269097Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4274958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4275989Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4276365Z 2025-12-04T09:43:15.4276518Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4276903Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4277345Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4278202Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4278858Z res = mod(**inputs) 2025-12-04T09:43:15.4279679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4280511Z outputs = self.model( 2025-12-04T09:43:15.4281282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4282214Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4282996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4283802Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4284486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4285222Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4285996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4286762Z return func(*args, **kwargs) 2025-12-04T09:43:15.4287518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4288402Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4288741Z 2025-12-04T09:43:15.4288889Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4289262Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4289631Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4290003Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4290376Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4290737Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4291106Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4291476Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4291894Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4292604Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4293253Z res = mod(**inputs) 2025-12-04T09:43:15.4293984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4294788Z outputs = self.model( 2025-12-04T09:43:15.4295532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4296339Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4297116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4297916Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4298620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4299355Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4300122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4301469Z return func(*args, **kwargs) 2025-12-04T09:43:15.4302269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4303146Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4303991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4304775Z return func(*args, **kwargs) 2025-12-04T09:43:15.4305562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4306567Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4307638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4308760Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4309138Z 2025-12-04T09:43:15.4309293Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4309680Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4310069Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4310457Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4310830Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4311209Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4311588Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4311962Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4312402Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4313135Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4313791Z res = mod(**inputs) 2025-12-04T09:43:15.4314553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4315377Z outputs = self.model( 2025-12-04T09:43:15.4316138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4316965Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4317765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4318594Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4319401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4320128Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4320899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4321668Z return func(*args, **kwargs) 2025-12-04T09:43:15.4322428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4323294Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4324126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4324895Z return func(*args, **kwargs) 2025-12-04T09:43:15.4325645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4326483Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4327396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4328388Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4328755Z 2025-12-04T09:43:15.4328893Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4329276Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4329707Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4330413Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4331049Z res = mod(**inputs) 2025-12-04T09:43:15.4331779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4337766Z outputs = self.model( 2025-12-04T09:43:15.4338529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4339462Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4340268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4341089Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4341868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4342625Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4343420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4344203Z return func(*args, **kwargs) 2025-12-04T09:43:15.4345102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4345993Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4346337Z 2025-12-04T09:43:15.4346487Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4346960Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4347533Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4347915Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4348292Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4348681Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4349069Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4349444Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4349879Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4350608Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4351258Z res = mod(**inputs) 2025-12-04T09:43:15.4352009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4352829Z outputs = self.model( 2025-12-04T09:43:15.4353592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4354418Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4355002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4355130Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4355611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4355752Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4356285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4356414Z return func(*args, **kwargs) 2025-12-04T09:43:15.4356972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4357153Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4357690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4357809Z return func(*args, **kwargs) 2025-12-04T09:43:15.4358383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4358688Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4359318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4359564Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4359580Z 2025-12-04T09:43:15.4359717Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4359854Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4359994Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4360190Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4360334Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4360460Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4360589Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4360783Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4360972Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4361378Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4361492Z res = mod(**inputs) 2025-12-04T09:43:15.4362048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4362173Z outputs = self.model( 2025-12-04T09:43:15.4362724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4362848Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4363412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4363533Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4364006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4364153Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4364670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4364799Z return func(*args, **kwargs) 2025-12-04T09:43:15.4365345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4365537Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4366065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4366184Z return func(*args, **kwargs) 2025-12-04T09:43:15.4366741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4366913Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4367537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4367781Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4367797Z 2025-12-04T09:43:15.4367931Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4368061Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4368261Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4368666Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4368792Z res = mod(**inputs) 2025-12-04T09:43:15.4369347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4369456Z outputs = self.model( 2025-12-04T09:43:15.4370020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4370147Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4370691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4370821Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4371292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4371434Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4371959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4372131Z return func(*args, **kwargs) 2025-12-04T09:43:15.4372687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4372979Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4372995Z 2025-12-04T09:43:15.4373142Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4373276Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4373412Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4373549Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4373678Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4373811Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4373946Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4374074Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4374260Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4374672Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4374779Z res = mod(**inputs) 2025-12-04T09:43:15.4375347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4375460Z outputs = self.model( 2025-12-04T09:43:15.4376012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4376142Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4376688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4376808Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4377283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4377418Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4377945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4378061Z return func(*args, **kwargs) 2025-12-04T09:43:15.4378610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4378793Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4379306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4379429Z return func(*args, **kwargs) 2025-12-04T09:43:15.4379979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4380145Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4380781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4381013Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4381028Z 2025-12-04T09:43:15.4381171Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4381307Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4381453Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4381581Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4381710Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4381848Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4381974Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4382114Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4382297Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4382696Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4382814Z res = mod(**inputs) 2025-12-04T09:43:15.4383438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4383553Z outputs = self.model( 2025-12-04T09:43:15.4384174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4384296Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4384853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4384971Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4385438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4385580Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4386090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4386209Z return func(*args, **kwargs) 2025-12-04T09:43:15.4386875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4387074Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4387766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4387890Z return func(*args, **kwargs) 2025-12-04T09:43:15.4388455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4388636Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4389280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4389530Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4389551Z 2025-12-04T09:43:15.4389692Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4389827Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4390030Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4390455Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4390567Z res = mod(**inputs) 2025-12-04T09:43:15.4391149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4391267Z outputs = self.model( 2025-12-04T09:43:15.4391839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4391966Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4392536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4392675Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4393154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4393294Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4393834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4393951Z return func(*args, **kwargs) 2025-12-04T09:43:15.4394533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4394804Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4394821Z 2025-12-04T09:43:15.4394960Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4395207Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4395347Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4395593Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4395732Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4395861Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4395998Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4396190Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4396382Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4396803Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4396913Z res = mod(**inputs) 2025-12-04T09:43:15.4397489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4397610Z outputs = self.model( 2025-12-04T09:43:15.4398180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4398319Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4398886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4399008Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4399554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4399701Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4400242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4400371Z return func(*args, **kwargs) 2025-12-04T09:43:15.4401347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4401539Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4402072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4402199Z return func(*args, **kwargs) 2025-12-04T09:43:15.4402767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4402941Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4403598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4403838Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4403854Z 2025-12-04T09:43:15.4403993Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4404140Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4404276Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4404409Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4404552Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4404690Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4404832Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4404961Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4405150Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4405584Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4405689Z res = mod(**inputs) 2025-12-04T09:43:15.4406258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4406388Z outputs = self.model( 2025-12-04T09:43:15.4406954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4407093Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4407658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4407904Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4408403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4408613Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4409152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4409284Z return func(*args, **kwargs) 2025-12-04T09:43:15.4409848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4410056Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4410589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4410705Z return func(*args, **kwargs) 2025-12-04T09:43:15.4411280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4411447Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4412089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4412337Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4412352Z 2025-12-04T09:43:15.4412489Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4412631Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4412819Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4413339Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4413452Z res = mod(**inputs) 2025-12-04T09:43:15.4413999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4414125Z outputs = self.model( 2025-12-04T09:43:15.4414672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4414799Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4415352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4415472Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4415940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4416078Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4416591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4416710Z return func(*args, **kwargs) 2025-12-04T09:43:15.4417250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4417463Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4417478Z 2025-12-04T09:43:15.4417620Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4417747Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4417877Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4418009Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4418137Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4418273Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4418396Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4418522Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4418714Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4419117Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4419282Z res = mod(**inputs) 2025-12-04T09:43:15.4419839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4419948Z outputs = self.model( 2025-12-04T09:43:15.4420558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4420681Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4421231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4421360Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4421828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4421960Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4422480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4422596Z return func(*args, **kwargs) 2025-12-04T09:43:15.4423146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4423317Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4423829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4423952Z return func(*args, **kwargs) 2025-12-04T09:43:15.4424499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4424670Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4425291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4425518Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4425537Z 2025-12-04T09:43:15.4425676Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4425803Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4425929Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4426067Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4426192Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4426327Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4426453Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4426578Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4426860Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4427440Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4427553Z res = mod(**inputs) 2025-12-04T09:43:15.4428141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4428263Z outputs = self.model( 2025-12-04T09:43:15.4428831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4428970Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4429537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4429675Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4430156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4430291Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4430836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4430952Z return func(*args, **kwargs) 2025-12-04T09:43:15.4431522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4431802Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4432392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4432520Z return func(*args, **kwargs) 2025-12-04T09:43:15.4433079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4433248Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4433893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4434130Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4434146Z 2025-12-04T09:43:15.4434293Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4434434Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4434619Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4435038Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4435151Z res = mod(**inputs) 2025-12-04T09:43:15.4435727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4435840Z outputs = self.model( 2025-12-04T09:43:15.4436401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4436534Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4437100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4437221Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4437714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4437848Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4438387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4438504Z return func(*args, **kwargs) 2025-12-04T09:43:15.4439062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4439388Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4439403Z 2025-12-04T09:43:15.4439536Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4439663Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4439796Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4439921Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4440053Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4440181Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4440309Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4440440Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4440618Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4441019Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4441129Z res = mod(**inputs) 2025-12-04T09:43:15.4441679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4445086Z outputs = self.model( 2025-12-04T09:43:15.4445663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4445789Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4446343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4446532Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4446997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4447201Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4447728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4447844Z return func(*args, **kwargs) 2025-12-04T09:43:15.4448393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4448597Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4449117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4449243Z return func(*args, **kwargs) 2025-12-04T09:43:15.4449790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4449963Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4450588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4450820Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4450836Z 2025-12-04T09:43:15.4450979Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4451109Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4451239Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4451372Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4451496Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4451631Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4451758Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4451884Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4452076Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4452477Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4452578Z res = mod(**inputs) 2025-12-04T09:43:15.4453149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4453260Z outputs = self.model( 2025-12-04T09:43:15.4453820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4453947Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4454494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4454623Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4455093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4455230Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4455756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4455875Z return func(*args, **kwargs) 2025-12-04T09:43:15.4456426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4456616Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4457297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4457426Z return func(*args, **kwargs) 2025-12-04T09:43:15.4458280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4458497Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4459147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4459445Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4459461Z 2025-12-04T09:43:15.4459610Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4459741Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4459928Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4460351Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4460465Z res = mod(**inputs) 2025-12-04T09:43:15.4461044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4461156Z outputs = self.model( 2025-12-04T09:43:15.4461718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4461852Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4462421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4462546Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4463038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4463176Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4463720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4463842Z return func(*args, **kwargs) 2025-12-04T09:43:15.4464405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4464630Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4464646Z 2025-12-04T09:43:15.4464785Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4464919Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4465057Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4465187Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4465327Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4465459Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4465589Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4465766Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4465955Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4466381Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4466487Z res = mod(**inputs) 2025-12-04T09:43:15.4467144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4467268Z outputs = self.model( 2025-12-04T09:43:15.4467836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4467970Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4468531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4468650Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4469141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4469331Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4469859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4469985Z return func(*args, **kwargs) 2025-12-04T09:43:15.4470543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 477, in forward 2025-12-04T09:43:15.4470756Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:15.4471361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4471479Z return func(*args, **kwargs) 2025-12-04T09:43:15.4472046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4472216Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4472868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4473105Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4473121Z 2025-12-04T09:43:15.4473258Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4473398Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4473529Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4473658Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4473795Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4473930Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4474063Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4474193Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4474378Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4474797Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4474907Z res = mod(**inputs) 2025-12-04T09:43:15.4475478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4475599Z outputs = self.model( 2025-12-04T09:43:15.4476164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4476307Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4476882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4477012Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4477513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4477655Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4478194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4478336Z return func(*args, **kwargs) 2025-12-04T09:43:15.4478995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 494, in forward 2025-12-04T09:43:15.4479203Z hidden_states, cross_attn_weights = self.encoder_attn( 2025-12-04T09:43:15.4479726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4479846Z return func(*args, **kwargs) 2025-12-04T09:43:15.4480413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 322, in forward 2025-12-04T09:43:15.4480583Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:15.4481222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:15.4481495Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:15.4481510Z 2025-12-04T09:43:15.4481650Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4481799Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4481985Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4482421Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4482547Z res = mod(**inputs) 2025-12-04T09:43:15.4483160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1401, in forward 2025-12-04T09:43:15.4483293Z outputs = self.model( 2025-12-04T09:43:15.4483851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1276, in forward 2025-12-04T09:43:15.4483979Z decoder_outputs = self.decoder( 2025-12-04T09:43:15.4484545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1130, in forward 2025-12-04T09:43:15.4484676Z layer_outputs = decoder_layer( 2025-12-04T09:43:15.4485147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:15.4485299Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:15.4485823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:15.4485958Z return func(*args, **kwargs) 2025-12-04T09:43:15.4486510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 508, in forward 2025-12-04T09:43:15.4486724Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:15.4486739Z 2025-12-04T09:43:15.4486891Z cudagraph partition due to non gpu ops 2025-12-04T09:43:15.4487079Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4487504Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4487616Z res = mod(**inputs) 2025-12-04T09:43:15.4488173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1419, in forward 2025-12-04T09:43:15.4488323Z lm_logits = self.lm_head(outputs[0]) 2025-12-04T09:43:15.4488338Z 2025-12-04T09:43:15.4488526Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:15.4488932Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:15.4489058Z res = mod(**inputs) 2025-12-04T09:43:15.4489617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/m2m_100/modeling_m2m_100.py", line 1426, in forward 2025-12-04T09:43:15.4489940Z masked_lm_loss = loss_fct(lm_logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:43:15.4489958Z 2025-12-04T09:43:25.6225405Z Compilation time (from dynamo_timed): 41.271880203 2025-12-04T09:43:25.6374064Z pass 2025-12-04T09:43:25.6375142Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:25.6376830Z TIMING: _recursive_pre_grad_passes:0.11562 _recursive_joint_graph_passes:1.32461 _recursive_post_grad_passes:0.16184 async_compile.wait:1.1216 code_gen:9.70377 inductor_compile:16.74761 backend_compile:32.76437 gc:0.00122 entire_frame_compile:41.27188 total_wall_time:41.27188 2025-12-04T09:43:25.6383832Z STATS: call_* op count: 906 | FakeTensorMode.__torch_dispatch__:46361 | FakeTensor.__torch_dispatch__:8011 | ProxyTorchDispatchMode.__torch_dispatch__:6566 2025-12-04T09:43:25.6384901Z Dynamo produced 1 graphs covering 906 ops with 0 graph breaks (0 unique) 2025-12-04T09:43:29.1278966Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:43:29.1281047Z import pynvml # type: ignore[import] 2025-12-04T09:43:34.0999696Z 2025-12-04T09:43:38.9365312Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:43:38.9365753Z loading model: 0it [00:04, ?it/s] 2025-12-04T09:43:38.9392368Z cpu eval MBartForCausalLM 2025-12-04T09:43:41.3564207Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:42.3953142Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:43.4586712Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:43:58.4030940Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4031631Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4032209Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4032640Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4033044Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4033443Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4033844Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4034236Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4034630Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4035039Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4035474Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4035942Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4036443Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4036911Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4037458Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4037909Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4038413Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4039007Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4039591Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4040164Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4041171Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4042191Z res = mod(**inputs) 2025-12-04T09:43:58.4043049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4044045Z outputs = self.model.decoder( 2025-12-04T09:43:58.4045021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4046140Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4046953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4047853Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4048760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4049696Z return func(*args, **kwargs) 2025-12-04T09:43:58.4050719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4051965Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4053189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4054254Z return func(*args, **kwargs) 2025-12-04T09:43:58.4055284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4056344Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4057415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4058868Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4059430Z 2025-12-04T09:43:58.4059628Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4060090Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4060708Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4061623Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4062506Z res = mod(**inputs) 2025-12-04T09:43:58.4063564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4064689Z outputs = self.model.decoder( 2025-12-04T09:43:58.4065691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4066654Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4068121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4069040Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4070083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4071047Z return func(*args, **kwargs) 2025-12-04T09:43:58.4071996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4073120Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4074203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4075131Z return self.act(input) 2025-12-04T09:43:58.4075480Z 2025-12-04T09:43:58.4075683Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4076166Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4081592Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4081979Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4082376Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4082772Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4083156Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4083555Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4084011Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4084762Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4085425Z res = mod(**inputs) 2025-12-04T09:43:58.4086530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4087697Z outputs = self.model.decoder( 2025-12-04T09:43:58.4088874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4089868Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4090702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4091603Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4092469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4093279Z return func(*args, **kwargs) 2025-12-04T09:43:58.4094084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4094949Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4095762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4096550Z return func(*args, **kwargs) 2025-12-04T09:43:58.4097329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4098295Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4099215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4100265Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4100637Z 2025-12-04T09:43:58.4101223Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4101809Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4102532Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4103294Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4103980Z res = mod(**inputs) 2025-12-04T09:43:58.4104735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4105589Z outputs = self.model.decoder( 2025-12-04T09:43:58.4106403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4107375Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4108109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4108892Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4109713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4110501Z return func(*args, **kwargs) 2025-12-04T09:43:58.4111303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4112231Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4113059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4113769Z return self.act(input) 2025-12-04T09:43:58.4113994Z 2025-12-04T09:43:58.4114138Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4114537Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4114922Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4115312Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4115705Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4116082Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4116475Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4116872Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4117320Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4118048Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4118822Z res = mod(**inputs) 2025-12-04T09:43:58.4119566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4120376Z outputs = self.model.decoder( 2025-12-04T09:43:58.4121163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4121984Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4122687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4123416Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4124198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4124978Z return func(*args, **kwargs) 2025-12-04T09:43:58.4125738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4126685Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4127512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4128293Z return func(*args, **kwargs) 2025-12-04T09:43:58.4129052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4129953Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4130952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4131954Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4132325Z 2025-12-04T09:43:58.4132465Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4132852Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4133289Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4133998Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4134649Z res = mod(**inputs) 2025-12-04T09:43:58.4135394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4136215Z outputs = self.model.decoder( 2025-12-04T09:43:58.4136982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4137787Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4138965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4139734Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4140541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4141349Z return func(*args, **kwargs) 2025-12-04T09:43:58.4142140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4143090Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4143932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4144663Z return self.act(input) 2025-12-04T09:43:58.4144871Z 2025-12-04T09:43:58.4145014Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4145414Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4145814Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4146211Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4146587Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4147089Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4147484Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4147863Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4148313Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4149056Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4149717Z res = mod(**inputs) 2025-12-04T09:43:58.4150479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4151326Z outputs = self.model.decoder( 2025-12-04T09:43:58.4152147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4152973Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4153705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4154475Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4155269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4156148Z return func(*args, **kwargs) 2025-12-04T09:43:58.4156945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4157834Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4158808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4159585Z return func(*args, **kwargs) 2025-12-04T09:43:58.4160455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4161319Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4162237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4163248Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4163617Z 2025-12-04T09:43:58.4163772Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4164153Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4164595Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4165320Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4165977Z res = mod(**inputs) 2025-12-04T09:43:58.4166708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4167529Z outputs = self.model.decoder( 2025-12-04T09:43:58.4168323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4169124Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4169834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4170592Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4171376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4172143Z return func(*args, **kwargs) 2025-12-04T09:43:58.4172927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4173833Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4174636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4175328Z return self.act(input) 2025-12-04T09:43:58.4175545Z 2025-12-04T09:43:58.4175685Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4176084Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4176460Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4176852Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4177238Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4177603Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4177981Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4178358Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4178796Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4179501Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4180148Z res = mod(**inputs) 2025-12-04T09:43:58.4180886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4181688Z outputs = self.model.decoder( 2025-12-04T09:43:58.4182475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4183323Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4184032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4184763Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4185548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4186360Z return func(*args, **kwargs) 2025-12-04T09:43:58.4187383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4188349Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4189203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4190003Z return func(*args, **kwargs) 2025-12-04T09:43:58.4190780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4191661Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4192619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4193658Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4194037Z 2025-12-04T09:43:58.4194181Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4208490Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4209005Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4209778Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4210467Z res = mod(**inputs) 2025-12-04T09:43:58.4211249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4212102Z outputs = self.model.decoder( 2025-12-04T09:43:58.4212921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4213761Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4214498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4215267Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4216076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4216890Z return func(*args, **kwargs) 2025-12-04T09:43:58.4217788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4218684Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4219488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4220207Z return self.act(input) 2025-12-04T09:43:58.4220409Z 2025-12-04T09:43:58.4220553Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4220952Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4221345Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4221717Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4222106Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4222492Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4222864Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4223255Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4223694Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4224420Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4225063Z res = mod(**inputs) 2025-12-04T09:43:58.4225803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4226848Z outputs = self.model.decoder( 2025-12-04T09:43:58.4227859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4228707Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4229510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4230282Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4231166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4231977Z return func(*args, **kwargs) 2025-12-04T09:43:58.4232778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4233670Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4234509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4235313Z return func(*args, **kwargs) 2025-12-04T09:43:58.4236111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4236985Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4237955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4239101Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4239478Z 2025-12-04T09:43:58.4239638Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4240016Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4240456Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4241179Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4241823Z res = mod(**inputs) 2025-12-04T09:43:58.4242569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4243397Z outputs = self.model.decoder( 2025-12-04T09:43:58.4244187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4244989Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4245702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4246450Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4247216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4247995Z return func(*args, **kwargs) 2025-12-04T09:43:58.4248774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4249674Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4250452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4251168Z return self.act(input) 2025-12-04T09:43:58.4251372Z 2025-12-04T09:43:58.4251532Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4251927Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4252307Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4252694Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4253081Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4253453Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4253842Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4254231Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4254706Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4255432Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4256084Z res = mod(**inputs) 2025-12-04T09:43:58.4256823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4257660Z outputs = self.model.decoder( 2025-12-04T09:43:58.4258447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4259327Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4260018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4260772Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4261556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4262339Z return func(*args, **kwargs) 2025-12-04T09:43:58.4263096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4269397Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4270262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4271056Z return func(*args, **kwargs) 2025-12-04T09:43:58.4271865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4272761Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4273723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4274742Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4275140Z 2025-12-04T09:43:58.4275290Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4275699Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4276154Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4276889Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4277569Z res = mod(**inputs) 2025-12-04T09:43:58.4278337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4279268Z outputs = self.model.decoder( 2025-12-04T09:43:58.4280058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4280871Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4281583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4282327Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4283113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4283888Z return func(*args, **kwargs) 2025-12-04T09:43:58.4284648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4285546Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4286349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4287061Z return self.act(input) 2025-12-04T09:43:58.4287263Z 2025-12-04T09:43:58.4287405Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4287804Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4288201Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4288649Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4289039Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4289430Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4289800Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4290190Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4290672Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4291401Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4292046Z res = mod(**inputs) 2025-12-04T09:43:58.4292863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4293683Z outputs = self.model.decoder( 2025-12-04T09:43:58.4294459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4295277Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4295984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4296730Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4297496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4298271Z return func(*args, **kwargs) 2025-12-04T09:43:58.4299039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4299906Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4300713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4302094Z return func(*args, **kwargs) 2025-12-04T09:43:58.4302904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4303779Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4304743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4305781Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4306165Z 2025-12-04T09:43:58.4306331Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4306722Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4307265Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4308021Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4308684Z res = mod(**inputs) 2025-12-04T09:43:58.4309450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4310306Z outputs = self.model.decoder( 2025-12-04T09:43:58.4311126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4311949Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4312681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4313455Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4314251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4315056Z return func(*args, **kwargs) 2025-12-04T09:43:58.4315862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4316788Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4317588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4318541Z return self.act(input) 2025-12-04T09:43:58.4318747Z 2025-12-04T09:43:58.4318907Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4319306Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4319688Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4320078Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4320518Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4320888Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4321275Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4321662Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4322187Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4322919Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4323582Z res = mod(**inputs) 2025-12-04T09:43:58.4324328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4325138Z outputs = self.model.decoder( 2025-12-04T09:43:58.4326078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4331641Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4332365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4333138Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4333954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4334763Z return func(*args, **kwargs) 2025-12-04T09:43:58.4335548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4336433Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4337273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4338185Z return func(*args, **kwargs) 2025-12-04T09:43:58.4338953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4339796Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4340725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4341725Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4342094Z 2025-12-04T09:43:58.4342250Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4342633Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4343082Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4343803Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4344446Z res = mod(**inputs) 2025-12-04T09:43:58.4345189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4346008Z outputs = self.model.decoder( 2025-12-04T09:43:58.4346877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4347862Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4348600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4349375Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4350170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4350968Z return func(*args, **kwargs) 2025-12-04T09:43:58.4351745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4352738Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4353538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4354292Z return self.act(input) 2025-12-04T09:43:58.4354500Z 2025-12-04T09:43:58.4354647Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4355036Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4355409Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4355848Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4356233Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4356602Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4356984Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4357361Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4357785Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4358521Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4359283Z res = mod(**inputs) 2025-12-04T09:43:58.4360004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4360802Z outputs = self.model.decoder( 2025-12-04T09:43:58.4361573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4362374Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4363057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4363787Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4364559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4365325Z return func(*args, **kwargs) 2025-12-04T09:43:58.4366066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4366918Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4367729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4368478Z return func(*args, **kwargs) 2025-12-04T09:43:58.4369238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4370081Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4370995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4371968Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4372342Z 2025-12-04T09:43:58.4372479Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4372852Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4373272Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4373974Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4374621Z res = mod(**inputs) 2025-12-04T09:43:58.4375343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4376135Z outputs = self.model.decoder( 2025-12-04T09:43:58.4376913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4377710Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4378397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4379162Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4379925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4380683Z return func(*args, **kwargs) 2025-12-04T09:43:58.4381418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4382330Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4383160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4383847Z return self.act(input) 2025-12-04T09:43:58.4384043Z 2025-12-04T09:43:58.4384177Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4384557Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4384933Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4385289Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4385666Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4386043Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4386399Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4386874Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4387475Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4388259Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4389037Z res = mod(**inputs) 2025-12-04T09:43:58.4389797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4390629Z outputs = self.model.decoder( 2025-12-04T09:43:58.4391413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4392242Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4392964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4393790Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4394572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4395361Z return func(*args, **kwargs) 2025-12-04T09:43:58.4396142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4397020Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4397841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4398632Z return func(*args, **kwargs) 2025-12-04T09:43:58.4399419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4400280Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4401616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4402647Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4403024Z 2025-12-04T09:43:58.4403173Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4403551Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4403984Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4404714Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4405362Z res = mod(**inputs) 2025-12-04T09:43:58.4406116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4406953Z outputs = self.model.decoder( 2025-12-04T09:43:58.4407746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4408674Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4409395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4410197Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4410972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4411758Z return func(*args, **kwargs) 2025-12-04T09:43:58.4412624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4413634Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4414397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4415089Z return self.act(input) 2025-12-04T09:43:58.4415287Z 2025-12-04T09:43:58.4415433Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4415807Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4416172Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4416542Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4416915Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4417270Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4417646Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4418022Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4418434Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4419154Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4419803Z res = mod(**inputs) 2025-12-04T09:43:58.4420529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4421327Z outputs = self.model.decoder( 2025-12-04T09:43:58.4422099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4422900Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4423580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4424316Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4425083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4425848Z return func(*args, **kwargs) 2025-12-04T09:43:58.4426589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 420, in forward 2025-12-04T09:43:58.4427720Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:43:58.4428557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4429332Z return func(*args, **kwargs) 2025-12-04T09:43:58.4430121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 267, in forward 2025-12-04T09:43:58.4430994Z attn_output, attn_weights = attention_interface( 2025-12-04T09:43:58.4431947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:43:58.4432956Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:43:58.4433348Z 2025-12-04T09:43:58.4433487Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4433870Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4434311Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4435037Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4435764Z res = mod(**inputs) 2025-12-04T09:43:58.4436512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1865, in forward 2025-12-04T09:43:58.4437332Z outputs = self.model.decoder( 2025-12-04T09:43:58.4438125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1116, in forward 2025-12-04T09:43:58.4439004Z layer_outputs = decoder_layer( 2025-12-04T09:43:58.4439822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:43:58.4440607Z return super().__call__(*args, **kwargs) 2025-12-04T09:43:58.4441382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:43:58.4442149Z return func(*args, **kwargs) 2025-12-04T09:43:58.4442888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 451, in forward 2025-12-04T09:43:58.4443769Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:43:58.4444545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:43:58.4445230Z return self.act(input) 2025-12-04T09:43:58.4445432Z 2025-12-04T09:43:58.4445564Z cudagraph partition due to non gpu ops 2025-12-04T09:43:58.4445988Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4446700Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4447326Z res = mod(**inputs) 2025-12-04T09:43:58.4448066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1881, in forward 2025-12-04T09:43:58.4448886Z logits = self.lm_head(outputs[0]) 2025-12-04T09:43:58.4449133Z 2025-12-04T09:43:58.4449340Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:43:58.4450053Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:43:58.4450761Z res = mod(**inputs) 2025-12-04T09:43:58.4451813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mbart/modeling_mbart.py", line 1887, in forward 2025-12-04T09:43:58.4452812Z loss = loss_fct(logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:43:58.4453227Z 2025-12-04T09:44:05.9104561Z Compilation time (from dynamo_timed): 20.629838085 2025-12-04T09:44:05.9430598Z pass 2025-12-04T09:44:05.9431440Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:44:05.9433153Z TIMING: _recursive_pre_grad_passes:0.04817 _recursive_joint_graph_passes:0.88405 _recursive_post_grad_passes:0.08947 async_compile.wait:1.20403 code_gen:6.55063 inductor_compile:10.57034 backend_compile:17.04755 gc:0.0019 entire_frame_compile:20.62984 total_wall_time:20.62984 2025-12-04T09:44:05.9435194Z STATS: call_* op count: 337 | FakeTensorMode.__torch_dispatch__:18975 | FakeTensor.__torch_dispatch__:3530 | ProxyTorchDispatchMode.__torch_dispatch__:2869 2025-12-04T09:44:05.9436230Z Dynamo produced 1 graphs covering 337 ops with 0 graph breaks (0 unique) 2025-12-04T09:44:08.8906228Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:44:08.8908114Z import pynvml # type: ignore[import] 2025-12-04T09:44:13.8672266Z 2025-12-04T09:44:18.3956239Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:44:18.3956700Z loading model: 0it [00:04, ?it/s] 2025-12-04T09:44:18.3990001Z cpu eval MT5ForConditionalGeneration 2025-12-04T09:44:19.2898397Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:44:19.7141979Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:44:20.1381662Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:44:44.0262978Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0264538Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0265452Z res = mod(**inputs) 2025-12-04T09:44:44.0266477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.0267538Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.0268548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0269532Z layer_outputs = layer_module( 2025-12-04T09:44:44.0270426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0271335Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0272159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0272962Z return func(*args, **kwargs) 2025-12-04T09:44:44.0273739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0274579Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0275425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0276390Z return func(*args, **kwargs) 2025-12-04T09:44:44.0277272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0278180Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0279236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0280126Z return func(*args, **kwargs) 2025-12-04T09:44:44.0280890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 424, in forward 2025-12-04T09:44:44.0281809Z position_bias = position_bias + causal_mask 2025-12-04T09:44:44.0282116Z 2025-12-04T09:44:44.0282342Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0283215Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0283898Z res = mod(**inputs) 2025-12-04T09:44:44.0284744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0285644Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0286420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0287325Z layer_outputs = layer_module( 2025-12-04T09:44:44.0288356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0289219Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0290024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0290937Z return func(*args, **kwargs) 2025-12-04T09:44:44.0291694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0292676Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0293625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0294630Z return func(*args, **kwargs) 2025-12-04T09:44:44.0295384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0296362Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0297254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0298136Z return func(*args, **kwargs) 2025-12-04T09:44:44.0298981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.0299870Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.0300263Z 2025-12-04T09:44:44.0300470Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0302067Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0302750Z res = mod(**inputs) 2025-12-04T09:44:44.0303662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0304562Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0305425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0306406Z layer_outputs = layer_module( 2025-12-04T09:44:44.0307218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0308113Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0309070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0309903Z return func(*args, **kwargs) 2025-12-04T09:44:44.0310762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0311777Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0312704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0313683Z return func(*args, **kwargs) 2025-12-04T09:44:44.0314789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0315990Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0316996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0317794Z return func(*args, **kwargs) 2025-12-04T09:44:44.0318563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.0319377Z key_states = self.k(current_states) 2025-12-04T09:44:44.0319643Z 2025-12-04T09:44:44.0319856Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0320591Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0321331Z res = mod(**inputs) 2025-12-04T09:44:44.0322071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0322890Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0323779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0324701Z layer_outputs = layer_module( 2025-12-04T09:44:44.0325704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0326903Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0328191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0329421Z return func(*args, **kwargs) 2025-12-04T09:44:44.0330445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0331386Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0332487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0333814Z return func(*args, **kwargs) 2025-12-04T09:44:44.0335000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0336362Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0337377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0338313Z return func(*args, **kwargs) 2025-12-04T09:44:44.0339155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.0340104Z value_states = self.v(current_states) 2025-12-04T09:44:44.0340441Z 2025-12-04T09:44:44.0340609Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0341077Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0341553Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0342319Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0342975Z res = mod(**inputs) 2025-12-04T09:44:44.0343681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0344712Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0345685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0346636Z layer_outputs = layer_module( 2025-12-04T09:44:44.0347641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0348648Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0349759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0350626Z return func(*args, **kwargs) 2025-12-04T09:44:44.0351535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0352645Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0354040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0355088Z return func(*args, **kwargs) 2025-12-04T09:44:44.0355857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0356694Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0357512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0358303Z return func(*args, **kwargs) 2025-12-04T09:44:44.0359076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.0359979Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.0360232Z 2025-12-04T09:44:44.0360423Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0361160Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0361823Z res = mod(**inputs) 2025-12-04T09:44:44.0362545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0363329Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0364100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0365005Z layer_outputs = layer_module( 2025-12-04T09:44:44.0365696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0366443Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0367271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0368053Z return func(*args, **kwargs) 2025-12-04T09:44:44.0368847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0369663Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0370468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0371336Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0372180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.0373003Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.0373301Z 2025-12-04T09:44:44.0373509Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0374215Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0374872Z res = mod(**inputs) 2025-12-04T09:44:44.0375597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0376390Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0377145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0378188Z layer_outputs = layer_module( 2025-12-04T09:44:44.0383950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0384706Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0385516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0386323Z return func(*args, **kwargs) 2025-12-04T09:44:44.0387208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0388048Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0388891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0389791Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0390685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.0391501Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.0391792Z 2025-12-04T09:44:44.0391990Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0392742Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0393404Z res = mod(**inputs) 2025-12-04T09:44:44.0394143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0394962Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0395754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0396555Z layer_outputs = layer_module( 2025-12-04T09:44:44.0397282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0398060Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0399032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0399813Z return func(*args, **kwargs) 2025-12-04T09:44:44.0400559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0402020Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0402843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0403856Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0404747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.0405583Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.0405870Z 2025-12-04T09:44:44.0406066Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0406812Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0407491Z res = mod(**inputs) 2025-12-04T09:44:44.0408205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0409028Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0409816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0410619Z layer_outputs = layer_module( 2025-12-04T09:44:44.0411332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0412096Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0413012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0413774Z return func(*args, **kwargs) 2025-12-04T09:44:44.0414510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0415321Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0416130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0416983Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0417847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.0418641Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.0418900Z 2025-12-04T09:44:44.0419101Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0419803Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0420455Z res = mod(**inputs) 2025-12-04T09:44:44.0421163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0421942Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0422702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0423490Z layer_outputs = layer_module( 2025-12-04T09:44:44.0424194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0424930Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0425709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0426487Z return func(*args, **kwargs) 2025-12-04T09:44:44.0427493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0428399Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0429217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0430019Z return func(*args, **kwargs) 2025-12-04T09:44:44.0430772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0431638Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0432506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0433310Z return func(*args, **kwargs) 2025-12-04T09:44:44.0434059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.0434874Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.0435136Z 2025-12-04T09:44:44.0435348Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0436072Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0436737Z res = mod(**inputs) 2025-12-04T09:44:44.0437467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0438282Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0439054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0439967Z layer_outputs = layer_module( 2025-12-04T09:44:44.0445817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0446580Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0447385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0448243Z return func(*args, **kwargs) 2025-12-04T09:44:44.0449008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0449815Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0450623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0451427Z return func(*args, **kwargs) 2025-12-04T09:44:44.0452179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0453105Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0453892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0454667Z return func(*args, **kwargs) 2025-12-04T09:44:44.0455398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.0456185Z key_states = self.k(current_states) 2025-12-04T09:44:44.0456435Z 2025-12-04T09:44:44.0456640Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0457359Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0458000Z res = mod(**inputs) 2025-12-04T09:44:44.0458717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0459510Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0460262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0461041Z layer_outputs = layer_module( 2025-12-04T09:44:44.0461744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0462553Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0463319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0464097Z return func(*args, **kwargs) 2025-12-04T09:44:44.0464841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0465661Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0466527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0467585Z return func(*args, **kwargs) 2025-12-04T09:44:44.0468364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0469174Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0469985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0470788Z return func(*args, **kwargs) 2025-12-04T09:44:44.0471559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.0472363Z value_states = self.v(current_states) 2025-12-04T09:44:44.0472648Z 2025-12-04T09:44:44.0472793Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0473201Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0473643Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0474395Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0475069Z res = mod(**inputs) 2025-12-04T09:44:44.0475787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0476601Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0477395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0478208Z layer_outputs = layer_module( 2025-12-04T09:44:44.0478919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0479789Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0480576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0481355Z return func(*args, **kwargs) 2025-12-04T09:44:44.0482094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0482896Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0483682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0484445Z return func(*args, **kwargs) 2025-12-04T09:44:44.0485178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0485973Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0486759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0487519Z return func(*args, **kwargs) 2025-12-04T09:44:44.0488260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.0489051Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.0489296Z 2025-12-04T09:44:44.0489486Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0490202Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0490852Z res = mod(**inputs) 2025-12-04T09:44:44.0491611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0492392Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0493161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0493977Z layer_outputs = layer_module( 2025-12-04T09:44:44.0494682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0495416Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0496254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0497035Z return func(*args, **kwargs) 2025-12-04T09:44:44.0497764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0498581Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0499384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0500247Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0519242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.0520319Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.0520632Z 2025-12-04T09:44:44.0520847Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0521574Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0522233Z res = mod(**inputs) 2025-12-04T09:44:44.0522961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0523764Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0524524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0525308Z layer_outputs = layer_module( 2025-12-04T09:44:44.0526017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0526751Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0527540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0528320Z return func(*args, **kwargs) 2025-12-04T09:44:44.0529068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0529871Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0530680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0531544Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0532407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.0533192Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.0533477Z 2025-12-04T09:44:44.0533669Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0534394Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0535042Z res = mod(**inputs) 2025-12-04T09:44:44.0535764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0536561Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0537332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0538223Z layer_outputs = layer_module( 2025-12-04T09:44:44.0538937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0539693Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0540460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0541295Z return func(*args, **kwargs) 2025-12-04T09:44:44.0542042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0542935Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0543730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0544595Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0545458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.0546270Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.0546545Z 2025-12-04T09:44:44.0546888Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0547800Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0548478Z res = mod(**inputs) 2025-12-04T09:44:44.0549202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0550028Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0550820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0551627Z layer_outputs = layer_module( 2025-12-04T09:44:44.0552337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0553107Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0553918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0554709Z return func(*args, **kwargs) 2025-12-04T09:44:44.0555477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0556320Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0557155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0558038Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0558924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.0559854Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.0560118Z 2025-12-04T09:44:44.0560322Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0561033Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0561687Z res = mod(**inputs) 2025-12-04T09:44:44.0562402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0563182Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0563946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0564728Z layer_outputs = layer_module( 2025-12-04T09:44:44.0565513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0566549Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0567367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0568237Z return func(*args, **kwargs) 2025-12-04T09:44:44.0568994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0569820Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0570672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0571476Z return func(*args, **kwargs) 2025-12-04T09:44:44.0572295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0573129Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0573949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0574749Z return func(*args, **kwargs) 2025-12-04T09:44:44.0575504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.0576321Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.0576585Z 2025-12-04T09:44:44.0576798Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0577531Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0578205Z res = mod(**inputs) 2025-12-04T09:44:44.0578941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0579875Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0580627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0581420Z layer_outputs = layer_module( 2025-12-04T09:44:44.0582129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0582864Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0583644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0584423Z return func(*args, **kwargs) 2025-12-04T09:44:44.0585162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0585941Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0586814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0587785Z return func(*args, **kwargs) 2025-12-04T09:44:44.0588533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0589358Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0590172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0590969Z return func(*args, **kwargs) 2025-12-04T09:44:44.0591719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.0592537Z key_states = self.k(current_states) 2025-12-04T09:44:44.0592796Z 2025-12-04T09:44:44.0593006Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0593753Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0594422Z res = mod(**inputs) 2025-12-04T09:44:44.0595163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0595977Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0596754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0597621Z layer_outputs = layer_module( 2025-12-04T09:44:44.0598348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0599115Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0600034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0601358Z return func(*args, **kwargs) 2025-12-04T09:44:44.0602292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0603111Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0603932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0604735Z return func(*args, **kwargs) 2025-12-04T09:44:44.0605504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0606311Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0607127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0607930Z return func(*args, **kwargs) 2025-12-04T09:44:44.0608696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.0609495Z value_states = self.v(current_states) 2025-12-04T09:44:44.0609790Z 2025-12-04T09:44:44.0609935Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0610342Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0610778Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0611523Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0612204Z res = mod(**inputs) 2025-12-04T09:44:44.0612945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0613742Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0614626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0615412Z layer_outputs = layer_module( 2025-12-04T09:44:44.0616104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0616861Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0617642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0618421Z return func(*args, **kwargs) 2025-12-04T09:44:44.0619155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0619959Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0620746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0621503Z return func(*args, **kwargs) 2025-12-04T09:44:44.0622245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0623049Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0623828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0624595Z return func(*args, **kwargs) 2025-12-04T09:44:44.0625334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.0626113Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.0626412Z 2025-12-04T09:44:44.0626615Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0627613Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0633299Z res = mod(**inputs) 2025-12-04T09:44:44.0634040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0634910Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0635696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0636606Z layer_outputs = layer_module( 2025-12-04T09:44:44.0637338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0638091Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0638902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0639707Z return func(*args, **kwargs) 2025-12-04T09:44:44.0640565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0641379Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0642187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0643048Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0643895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.0644723Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.0645013Z 2025-12-04T09:44:44.0645209Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0645912Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0646542Z res = mod(**inputs) 2025-12-04T09:44:44.0647237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0648011Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0648746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0649512Z layer_outputs = layer_module( 2025-12-04T09:44:44.0650200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0650934Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0651688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0652453Z return func(*args, **kwargs) 2025-12-04T09:44:44.0653183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0653974Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0654762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0655611Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0656449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.0657218Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.0657480Z 2025-12-04T09:44:44.0657666Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0658360Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0658995Z res = mod(**inputs) 2025-12-04T09:44:44.0659684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0660494Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0661242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0661997Z layer_outputs = layer_module( 2025-12-04T09:44:44.0662712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0663437Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0664258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0665010Z return func(*args, **kwargs) 2025-12-04T09:44:44.0665742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0666539Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0667611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0668482Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0669353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.0670172Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.0670448Z 2025-12-04T09:44:44.0670636Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0671363Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0672013Z res = mod(**inputs) 2025-12-04T09:44:44.0672720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0673503Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0674272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0675065Z layer_outputs = layer_module( 2025-12-04T09:44:44.0675756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0676506Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0677290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0678073Z return func(*args, **kwargs) 2025-12-04T09:44:44.0678815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0679721Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0680511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0681351Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0682199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.0682967Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.0683219Z 2025-12-04T09:44:44.0683406Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0684098Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0684727Z res = mod(**inputs) 2025-12-04T09:44:44.0685419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0686193Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0686924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0687687Z layer_outputs = layer_module( 2025-12-04T09:44:44.0688417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0689135Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0689897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0696216Z return func(*args, **kwargs) 2025-12-04T09:44:44.0696977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0697780Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0698647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0699428Z return func(*args, **kwargs) 2025-12-04T09:44:44.0700165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0701365Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0702163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0702948Z return func(*args, **kwargs) 2025-12-04T09:44:44.0703684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.0704488Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.0704749Z 2025-12-04T09:44:44.0704953Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0705687Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0706332Z res = mod(**inputs) 2025-12-04T09:44:44.0707162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0707960Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0708724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0709516Z layer_outputs = layer_module( 2025-12-04T09:44:44.0710215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0710972Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0711742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0712523Z return func(*args, **kwargs) 2025-12-04T09:44:44.0713274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0714064Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0714854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0715636Z return func(*args, **kwargs) 2025-12-04T09:44:44.0716378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0717174Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0717963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0718847Z return func(*args, **kwargs) 2025-12-04T09:44:44.0719560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.0720323Z key_states = self.k(current_states) 2025-12-04T09:44:44.0720581Z 2025-12-04T09:44:44.0720763Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0721461Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0722084Z res = mod(**inputs) 2025-12-04T09:44:44.0722772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0723640Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0724385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0725195Z layer_outputs = layer_module( 2025-12-04T09:44:44.0725881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0726613Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0727443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0728212Z return func(*args, **kwargs) 2025-12-04T09:44:44.0728946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0729726Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0730494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0731255Z return func(*args, **kwargs) 2025-12-04T09:44:44.0731984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0732756Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0733526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0734290Z return func(*args, **kwargs) 2025-12-04T09:44:44.0735012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.0735771Z value_states = self.v(current_states) 2025-12-04T09:44:44.0736031Z 2025-12-04T09:44:44.0736164Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0736542Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0736951Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0737654Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0738288Z res = mod(**inputs) 2025-12-04T09:44:44.0738988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0739752Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0740511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0741284Z layer_outputs = layer_module( 2025-12-04T09:44:44.0741969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0742684Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0743443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0744198Z return func(*args, **kwargs) 2025-12-04T09:44:44.0744906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0745687Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0746457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0747493Z return func(*args, **kwargs) 2025-12-04T09:44:44.0748235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0749036Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0749823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0750586Z return func(*args, **kwargs) 2025-12-04T09:44:44.0751398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.0752193Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.0752438Z 2025-12-04T09:44:44.0752674Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0753564Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0754223Z res = mod(**inputs) 2025-12-04T09:44:44.0755016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0755820Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0756612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0757489Z layer_outputs = layer_module( 2025-12-04T09:44:44.0758212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0758974Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0759782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0760583Z return func(*args, **kwargs) 2025-12-04T09:44:44.0761349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0762155Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0762971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0763766Z return func(*args, **kwargs) 2025-12-04T09:44:44.0764507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 489, in forward 2025-12-04T09:44:44.0765522Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:44:44.0765910Z 2025-12-04T09:44:44.0766096Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0766813Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0767449Z res = mod(**inputs) 2025-12-04T09:44:44.0768164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0768953Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0769708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0770492Z layer_outputs = layer_module( 2025-12-04T09:44:44.0771194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0771941Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0772703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0773480Z return func(*args, **kwargs) 2025-12-04T09:44:44.0774215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0775026Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0775811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0776673Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0777533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.0778341Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.0778653Z 2025-12-04T09:44:44.0778839Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0779608Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0780265Z res = mod(**inputs) 2025-12-04T09:44:44.0780960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0781753Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0782557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0783352Z layer_outputs = layer_module( 2025-12-04T09:44:44.0784103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0784847Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0785628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0786391Z return func(*args, **kwargs) 2025-12-04T09:44:44.0787416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0788253Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0789083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0789970Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0790856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.0791692Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.0791962Z 2025-12-04T09:44:44.0792154Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0792900Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0793578Z res = mod(**inputs) 2025-12-04T09:44:44.0794308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0795105Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0795892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0796694Z layer_outputs = layer_module( 2025-12-04T09:44:44.0797389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0798138Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0798918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0799799Z return func(*args, **kwargs) 2025-12-04T09:44:44.0800520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0801853Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0802663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0803544Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0804403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.0805223Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.0805501Z 2025-12-04T09:44:44.0805697Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0806420Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0807080Z res = mod(**inputs) 2025-12-04T09:44:44.0807799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0808602Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0809459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0810253Z layer_outputs = layer_module( 2025-12-04T09:44:44.0810956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0811744Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0812523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0813306Z return func(*args, **kwargs) 2025-12-04T09:44:44.0815548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0816523Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0817331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0818207Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0819077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.0819875Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.0820152Z 2025-12-04T09:44:44.0820342Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0821065Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0821708Z res = mod(**inputs) 2025-12-04T09:44:44.0822434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0823235Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0824006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0824802Z layer_outputs = layer_module( 2025-12-04T09:44:44.0825504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0826274Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0827139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0827937Z return func(*args, **kwargs) 2025-12-04T09:44:44.0828691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0829511Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0830300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0831075Z return func(*args, **kwargs) 2025-12-04T09:44:44.0831820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0832630Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0833413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0834194Z return func(*args, **kwargs) 2025-12-04T09:44:44.0834935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.0835724Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.0835987Z 2025-12-04T09:44:44.0836176Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0836912Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0837562Z res = mod(**inputs) 2025-12-04T09:44:44.0838267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0839240Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0839992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0840749Z layer_outputs = layer_module( 2025-12-04T09:44:44.0841435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0842192Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0842952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0843757Z return func(*args, **kwargs) 2025-12-04T09:44:44.0844485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0845267Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0846030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0846784Z return func(*args, **kwargs) 2025-12-04T09:44:44.0847499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0848283Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0849038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0849799Z return func(*args, **kwargs) 2025-12-04T09:44:44.0850517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.0851290Z key_states = self.k(current_states) 2025-12-04T09:44:44.0851532Z 2025-12-04T09:44:44.0851717Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0852410Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0853057Z res = mod(**inputs) 2025-12-04T09:44:44.0853733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0854499Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0855254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0856018Z layer_outputs = layer_module( 2025-12-04T09:44:44.0856695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0857427Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0858193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0858952Z return func(*args, **kwargs) 2025-12-04T09:44:44.0859663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0860444Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0861211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0861964Z return func(*args, **kwargs) 2025-12-04T09:44:44.0862693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0863481Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0864256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0865012Z return func(*args, **kwargs) 2025-12-04T09:44:44.0865747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.0866529Z value_states = self.v(current_states) 2025-12-04T09:44:44.0866870Z 2025-12-04T09:44:44.0867056Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0867619Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0868048Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0868755Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0869436Z res = mod(**inputs) 2025-12-04T09:44:44.0870142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0870943Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0871755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0872539Z layer_outputs = layer_module( 2025-12-04T09:44:44.0873240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0873991Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0874782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0875559Z return func(*args, **kwargs) 2025-12-04T09:44:44.0876304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0877107Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0877969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0883562Z return func(*args, **kwargs) 2025-12-04T09:44:44.0884310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0885102Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0885891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0886676Z return func(*args, **kwargs) 2025-12-04T09:44:44.0887411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.0888197Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.0888449Z 2025-12-04T09:44:44.0888640Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0889355Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0890110Z res = mod(**inputs) 2025-12-04T09:44:44.0890805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0891582Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0892321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0893083Z layer_outputs = layer_module( 2025-12-04T09:44:44.0893773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0894510Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0895266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0896016Z return func(*args, **kwargs) 2025-12-04T09:44:44.0896742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0897533Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0898314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0899170Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0900003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.0901409Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.0901719Z 2025-12-04T09:44:44.0901972Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0902696Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0903448Z res = mod(**inputs) 2025-12-04T09:44:44.0904158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0904966Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0905807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0906588Z layer_outputs = layer_module( 2025-12-04T09:44:44.0907451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0908206Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0908995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0909767Z return func(*args, **kwargs) 2025-12-04T09:44:44.0910519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0911344Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0912158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0913027Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0913903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.0914701Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.0914967Z 2025-12-04T09:44:44.0915168Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0915882Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0916534Z res = mod(**inputs) 2025-12-04T09:44:44.0917254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0918047Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0918914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0919677Z layer_outputs = layer_module( 2025-12-04T09:44:44.0920372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0921090Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0921854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0922612Z return func(*args, **kwargs) 2025-12-04T09:44:44.0923323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0924131Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0924923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0925775Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0926613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.0927404Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.0927674Z 2025-12-04T09:44:44.0927865Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0928567Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0929266Z res = mod(**inputs) 2025-12-04T09:44:44.0929964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0930740Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0931476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0932279Z layer_outputs = layer_module( 2025-12-04T09:44:44.0932964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0933769Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0934526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0935296Z return func(*args, **kwargs) 2025-12-04T09:44:44.0936021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.0936811Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.0937603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.0938452Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.0939318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.0940122Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.0940471Z 2025-12-04T09:44:44.0945977Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0946853Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0947548Z res = mod(**inputs) 2025-12-04T09:44:44.0948272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0949094Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0949885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0950679Z layer_outputs = layer_module( 2025-12-04T09:44:44.0951403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0952180Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0952990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0953782Z return func(*args, **kwargs) 2025-12-04T09:44:44.0954555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0955379Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0956174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0956976Z return func(*args, **kwargs) 2025-12-04T09:44:44.0957746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0958679Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0959457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0960234Z return func(*args, **kwargs) 2025-12-04T09:44:44.0960987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.0961778Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.0962038Z 2025-12-04T09:44:44.0962228Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0962946Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0963668Z res = mod(**inputs) 2025-12-04T09:44:44.0964363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0965150Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0965911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0966732Z layer_outputs = layer_module( 2025-12-04T09:44:44.0967414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0968215Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0969000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0969763Z return func(*args, **kwargs) 2025-12-04T09:44:44.0970500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0971300Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0972090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0972849Z return func(*args, **kwargs) 2025-12-04T09:44:44.0973598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0974400Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0975169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0975940Z return func(*args, **kwargs) 2025-12-04T09:44:44.0976683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.0977471Z key_states = self.k(current_states) 2025-12-04T09:44:44.0977722Z 2025-12-04T09:44:44.0977912Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0978633Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0979279Z res = mod(**inputs) 2025-12-04T09:44:44.0979993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0980772Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0981535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0982322Z layer_outputs = layer_module( 2025-12-04T09:44:44.0983005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.0983754Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.0984531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0985305Z return func(*args, **kwargs) 2025-12-04T09:44:44.0986029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.0986913Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.0987893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0988677Z return func(*args, **kwargs) 2025-12-04T09:44:44.0989447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.0990269Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.0991080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.0991870Z return func(*args, **kwargs) 2025-12-04T09:44:44.0992678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.0993492Z value_states = self.v(current_states) 2025-12-04T09:44:44.0993759Z 2025-12-04T09:44:44.0993921Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0994312Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.0994799Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.0995537Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.0996193Z res = mod(**inputs) 2025-12-04T09:44:44.0996979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.0997791Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.0998576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.0999483Z layer_outputs = layer_module( 2025-12-04T09:44:44.1000183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1001312Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1002279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1003246Z return func(*args, **kwargs) 2025-12-04T09:44:44.1004017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1004847Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1005645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1006442Z return func(*args, **kwargs) 2025-12-04T09:44:44.1007209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1008088Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1008906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1009707Z return func(*args, **kwargs) 2025-12-04T09:44:44.1010474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1011265Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1011533Z 2025-12-04T09:44:44.1011731Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1012468Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1013127Z res = mod(**inputs) 2025-12-04T09:44:44.1013862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1014791Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1015554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1016322Z layer_outputs = layer_module( 2025-12-04T09:44:44.1017020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1017765Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1018548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1019313Z return func(*args, **kwargs) 2025-12-04T09:44:44.1020055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1020854Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1021619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1022496Z return func(*args, **kwargs) 2025-12-04T09:44:44.1023240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 489, in forward 2025-12-04T09:44:44.1024138Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:44:44.1024555Z 2025-12-04T09:44:44.1024745Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1025469Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1026133Z res = mod(**inputs) 2025-12-04T09:44:44.1027003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1027991Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1028783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1029595Z layer_outputs = layer_module( 2025-12-04T09:44:44.1030305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1031086Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1031901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1032713Z return func(*args, **kwargs) 2025-12-04T09:44:44.1033465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1034315Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1035139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1036016Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1036899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1037746Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1038054Z 2025-12-04T09:44:44.1038265Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1038989Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1039753Z res = mod(**inputs) 2025-12-04T09:44:44.1040458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1041241Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1042006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1042790Z layer_outputs = layer_module( 2025-12-04T09:44:44.1043487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1044219Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1044999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1045773Z return func(*args, **kwargs) 2025-12-04T09:44:44.1046519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1047317Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1048128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1048992Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1049831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1050629Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1050944Z 2025-12-04T09:44:44.1051132Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1051853Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1052498Z res = mod(**inputs) 2025-12-04T09:44:44.1053209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1054034Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1054785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1055645Z layer_outputs = layer_module( 2025-12-04T09:44:44.1056348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1057093Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1057858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1058634Z return func(*args, **kwargs) 2025-12-04T09:44:44.1059373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1060186Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1060972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1061833Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1062693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1063480Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1063766Z 2025-12-04T09:44:44.1063953Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1064667Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1065401Z res = mod(**inputs) 2025-12-04T09:44:44.1066397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1067304Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1068101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1068894Z layer_outputs = layer_module( 2025-12-04T09:44:44.1069631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1070402Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1071202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1071983Z return func(*args, **kwargs) 2025-12-04T09:44:44.1072742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1073584Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1074404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1075286Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1076169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1076973Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1090638Z 2025-12-04T09:44:44.1091409Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1092151Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1092792Z res = mod(**inputs) 2025-12-04T09:44:44.1093513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1094396Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1095164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1095934Z layer_outputs = layer_module( 2025-12-04T09:44:44.1096680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1097430Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1098255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1099026Z return func(*args, **kwargs) 2025-12-04T09:44:44.1099756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1100544Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1101987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1102784Z return func(*args, **kwargs) 2025-12-04T09:44:44.1103548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1104358Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1105158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1105942Z return func(*args, **kwargs) 2025-12-04T09:44:44.1106698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1107651Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1107924Z 2025-12-04T09:44:44.1108116Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1108860Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1109520Z res = mod(**inputs) 2025-12-04T09:44:44.1110239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1111050Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1111828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1112615Z layer_outputs = layer_module( 2025-12-04T09:44:44.1113333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1114083Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1114878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1115650Z return func(*args, **kwargs) 2025-12-04T09:44:44.1116407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1117223Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1118015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1118905Z return func(*args, **kwargs) 2025-12-04T09:44:44.1119640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1120433Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1121203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1121968Z return func(*args, **kwargs) 2025-12-04T09:44:44.1122700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1123561Z key_states = self.k(current_states) 2025-12-04T09:44:44.1123820Z 2025-12-04T09:44:44.1124007Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1124717Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1125363Z res = mod(**inputs) 2025-12-04T09:44:44.1126125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1126260Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1126858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1126984Z layer_outputs = layer_module( 2025-12-04T09:44:44.1127470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1127658Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1133310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1133437Z return func(*args, **kwargs) 2025-12-04T09:44:44.1133977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1134140Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1134674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1134807Z return func(*args, **kwargs) 2025-12-04T09:44:44.1135340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1135487Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1136034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1136157Z return func(*args, **kwargs) 2025-12-04T09:44:44.1136691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1136838Z value_states = self.v(current_states) 2025-12-04T09:44:44.1136854Z 2025-12-04T09:44:44.1136995Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1137151Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1137341Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1137756Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1137887Z res = mod(**inputs) 2025-12-04T09:44:44.1138424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1138554Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1139108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1139231Z layer_outputs = layer_module( 2025-12-04T09:44:44.1139727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1139867Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1140510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1140642Z return func(*args, **kwargs) 2025-12-04T09:44:44.1141156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1141304Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1141818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1141933Z return func(*args, **kwargs) 2025-12-04T09:44:44.1142508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1142649Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1143159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1143327Z return func(*args, **kwargs) 2025-12-04T09:44:44.1143842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1143988Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1144003Z 2025-12-04T09:44:44.1144250Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1144653Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1144778Z res = mod(**inputs) 2025-12-04T09:44:44.1145299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1145429Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1145962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1146086Z layer_outputs = layer_module( 2025-12-04T09:44:44.1146570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1146708Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1147520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1147661Z return func(*args, **kwargs) 2025-12-04T09:44:44.1148196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1148372Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1148913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1149131Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1149668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1149841Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1149857Z 2025-12-04T09:44:44.1150049Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1150479Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1150595Z res = mod(**inputs) 2025-12-04T09:44:44.1151153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1151286Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1151822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1151966Z layer_outputs = layer_module( 2025-12-04T09:44:44.1152451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1152592Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1153138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1153260Z return func(*args, **kwargs) 2025-12-04T09:44:44.1153809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1153968Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1154497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1154719Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1155287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1155444Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1155459Z 2025-12-04T09:44:44.1155649Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1156109Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1156233Z res = mod(**inputs) 2025-12-04T09:44:44.1156830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1156966Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1157511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1157634Z layer_outputs = layer_module( 2025-12-04T09:44:44.1158133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1158272Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1158801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1158936Z return func(*args, **kwargs) 2025-12-04T09:44:44.1159560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1159726Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1160239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1160442Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1160968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1161112Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1161128Z 2025-12-04T09:44:44.1161313Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1161732Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1161843Z res = mod(**inputs) 2025-12-04T09:44:44.1162378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1162507Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1163030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1163166Z layer_outputs = layer_module( 2025-12-04T09:44:44.1163632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1163766Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1164295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1164410Z return func(*args, **kwargs) 2025-12-04T09:44:44.1164937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1165093Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1165600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1165821Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1166336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1166484Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1166500Z 2025-12-04T09:44:44.1166685Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1167117Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1167237Z res = mod(**inputs) 2025-12-04T09:44:44.1167756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1167910Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1168440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1168562Z layer_outputs = layer_module( 2025-12-04T09:44:44.1169097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1169235Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1169752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1169887Z return func(*args, **kwargs) 2025-12-04T09:44:44.1170403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1170550Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1171071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1171195Z return func(*args, **kwargs) 2025-12-04T09:44:44.1171717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1171861Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1172377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1172504Z return func(*args, **kwargs) 2025-12-04T09:44:44.1173017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1173160Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1173175Z 2025-12-04T09:44:44.1173363Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1173767Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1173892Z res = mod(**inputs) 2025-12-04T09:44:44.1174417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1174544Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1175082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1175203Z layer_outputs = layer_module( 2025-12-04T09:44:44.1175682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1175817Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1176338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1176468Z return func(*args, **kwargs) 2025-12-04T09:44:44.1176984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1177143Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1177656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1177778Z return func(*args, **kwargs) 2025-12-04T09:44:44.1178304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1178443Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1178956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1179132Z return func(*args, **kwargs) 2025-12-04T09:44:44.1179647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1179791Z key_states = self.k(current_states) 2025-12-04T09:44:44.1179835Z 2025-12-04T09:44:44.1180018Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1180422Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1180545Z res = mod(**inputs) 2025-12-04T09:44:44.1181118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1181247Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1181780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1181902Z layer_outputs = layer_module( 2025-12-04T09:44:44.1182385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1182522Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1183037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1183174Z return func(*args, **kwargs) 2025-12-04T09:44:44.1183687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1183843Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1184360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1184480Z return func(*args, **kwargs) 2025-12-04T09:44:44.1185014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1185158Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1185671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1185806Z return func(*args, **kwargs) 2025-12-04T09:44:44.1186320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1186467Z value_states = self.v(current_states) 2025-12-04T09:44:44.1186482Z 2025-12-04T09:44:44.1186619Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1186864Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1187071Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1187656Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1187766Z res = mod(**inputs) 2025-12-04T09:44:44.1188322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1188453Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1189001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1189126Z layer_outputs = layer_module( 2025-12-04T09:44:44.1189611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1189762Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1190372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1190495Z return func(*args, **kwargs) 2025-12-04T09:44:44.1195843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1195988Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1196583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1196705Z return func(*args, **kwargs) 2025-12-04T09:44:44.1197235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 478, in forward 2025-12-04T09:44:44.1197468Z normed_hidden_states = self.layer_norm(hidden_states) 2025-12-04T09:44:44.1198001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 147, in forward 2025-12-04T09:44:44.1198203Z return self.weight * hidden_states 2025-12-04T09:44:44.1198220Z 2025-12-04T09:44:44.1198412Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1198833Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1198952Z res = mod(**inputs) 2025-12-04T09:44:44.1199495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1199628Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1200170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1200298Z layer_outputs = layer_module( 2025-12-04T09:44:44.1201188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1201339Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1201883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1202020Z return func(*args, **kwargs) 2025-12-04T09:44:44.1202555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1202700Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1203248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1203376Z return func(*args, **kwargs) 2025-12-04T09:44:44.1203913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1204059Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1204582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1204711Z return func(*args, **kwargs) 2025-12-04T09:44:44.1205242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1205380Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1205397Z 2025-12-04T09:44:44.1205585Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1206003Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1206118Z res = mod(**inputs) 2025-12-04T09:44:44.1206656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1206790Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1207332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1207456Z layer_outputs = layer_module( 2025-12-04T09:44:44.1207953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1208092Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1208620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1208752Z return func(*args, **kwargs) 2025-12-04T09:44:44.1209382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1209526Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1210068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1210236Z return func(*args, **kwargs) 2025-12-04T09:44:44.1210783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1211004Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1211535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1211663Z return func(*args, **kwargs) 2025-12-04T09:44:44.1212195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1212338Z key_states = self.k(current_states) 2025-12-04T09:44:44.1212353Z 2025-12-04T09:44:44.1212542Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1213058Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1213180Z res = mod(**inputs) 2025-12-04T09:44:44.1213696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1213823Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1214357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1214476Z layer_outputs = layer_module( 2025-12-04T09:44:44.1214956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1215090Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1215604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1215731Z return func(*args, **kwargs) 2025-12-04T09:44:44.1216243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1216383Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1216908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1217026Z return func(*args, **kwargs) 2025-12-04T09:44:44.1217556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1217694Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1218203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1218332Z return func(*args, **kwargs) 2025-12-04T09:44:44.1218842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1218976Z value_states = self.v(current_states) 2025-12-04T09:44:44.1218991Z 2025-12-04T09:44:44.1219126Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1219255Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1219449Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1219850Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1219953Z res = mod(**inputs) 2025-12-04T09:44:44.1220479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1220601Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1221129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1221282Z layer_outputs = layer_module( 2025-12-04T09:44:44.1221747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1221892Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1222440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1222555Z return func(*args, **kwargs) 2025-12-04T09:44:44.1223129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1223269Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1223789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1223906Z return func(*args, **kwargs) 2025-12-04T09:44:44.1224420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1224570Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1225082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1225201Z return func(*args, **kwargs) 2025-12-04T09:44:44.1225727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1225854Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1225873Z 2025-12-04T09:44:44.1226066Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1226467Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1226572Z res = mod(**inputs) 2025-12-04T09:44:44.1227379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1227515Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1228066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1228188Z layer_outputs = layer_module( 2025-12-04T09:44:44.1228675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1228820Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1229351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1229473Z return func(*args, **kwargs) 2025-12-04T09:44:44.1230008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1230153Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1230693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1230811Z return func(*args, **kwargs) 2025-12-04T09:44:44.1231340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1231498Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1232028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1232149Z return func(*args, **kwargs) 2025-12-04T09:44:44.1232690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1232822Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1232837Z 2025-12-04T09:44:44.1233036Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1233488Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1233600Z res = mod(**inputs) 2025-12-04T09:44:44.1234142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1234267Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1234845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1234968Z layer_outputs = layer_module( 2025-12-04T09:44:44.1235529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1235680Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1236209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1236330Z return func(*args, **kwargs) 2025-12-04T09:44:44.1236869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1237008Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1237553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1237673Z return func(*args, **kwargs) 2025-12-04T09:44:44.1238202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1238361Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1238891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1239023Z return func(*args, **kwargs) 2025-12-04T09:44:44.1239647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1239775Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1239790Z 2025-12-04T09:44:44.1239983Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1240380Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1240484Z res = mod(**inputs) 2025-12-04T09:44:44.1241015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1241138Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1241672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1241792Z layer_outputs = layer_module( 2025-12-04T09:44:44.1242254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1242399Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1242914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1243028Z return func(*args, **kwargs) 2025-12-04T09:44:44.1243551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1243694Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1244211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1244329Z return func(*args, **kwargs) 2025-12-04T09:44:44.1244838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 489, in forward 2025-12-04T09:44:44.1245086Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:44:44.1245101Z 2025-12-04T09:44:44.1245286Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1245730Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1245836Z res = mod(**inputs) 2025-12-04T09:44:44.1246356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1246490Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1247036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1247156Z layer_outputs = layer_module( 2025-12-04T09:44:44.1247697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1247833Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1248356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1248471Z return func(*args, **kwargs) 2025-12-04T09:44:44.1248983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1249145Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1249655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1249863Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1250386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1250553Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1250568Z 2025-12-04T09:44:44.1250764Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1251162Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1251270Z res = mod(**inputs) 2025-12-04T09:44:44.1251796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1251917Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1252446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1252604Z layer_outputs = layer_module( 2025-12-04T09:44:44.1253449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1253601Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1254136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1254255Z return func(*args, **kwargs) 2025-12-04T09:44:44.1254790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1254947Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1255483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1255694Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1256221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1256375Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1256391Z 2025-12-04T09:44:44.1256575Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1257003Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1257113Z res = mod(**inputs) 2025-12-04T09:44:44.1257709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1257848Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1258432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1258552Z layer_outputs = layer_module( 2025-12-04T09:44:44.1259042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1259209Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1259753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1259928Z return func(*args, **kwargs) 2025-12-04T09:44:44.1260456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1260625Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1261154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1261366Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1261906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1262052Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1262071Z 2025-12-04T09:44:44.1262270Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1262679Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1262788Z res = mod(**inputs) 2025-12-04T09:44:44.1263344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1750, in forward 2025-12-04T09:44:44.1263470Z encoder_outputs = self.encoder( 2025-12-04T09:44:44.1264020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1264147Z layer_outputs = layer_module( 2025-12-04T09:44:44.1264628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1264777Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1265312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1265433Z return func(*args, **kwargs) 2025-12-04T09:44:44.1265974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1266139Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1266676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1266986Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1267520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1267669Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1267684Z 2025-12-04T09:44:44.1267873Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1268294Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1268411Z res = mod(**inputs) 2025-12-04T09:44:44.1268958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1269100Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1269634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1269757Z layer_outputs = layer_module( 2025-12-04T09:44:44.1270243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1270425Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1270962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1271081Z return func(*args, **kwargs) 2025-12-04T09:44:44.1271609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1271792Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1272381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1272501Z return func(*args, **kwargs) 2025-12-04T09:44:44.1273037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1273186Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1273717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1273837Z return func(*args, **kwargs) 2025-12-04T09:44:44.1274368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1274510Z key_states = self.k(current_states) 2025-12-04T09:44:44.1274528Z 2025-12-04T09:44:44.1274716Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1275139Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1275252Z res = mod(**inputs) 2025-12-04T09:44:44.1275790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1275929Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1276462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1276591Z layer_outputs = layer_module( 2025-12-04T09:44:44.1277086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1277223Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1277764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1277885Z return func(*args, **kwargs) 2025-12-04T09:44:44.1278419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1278573Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1279197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1279312Z return func(*args, **kwargs) 2025-12-04T09:44:44.1279830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1279975Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1280501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1280612Z return func(*args, **kwargs) 2025-12-04T09:44:44.1281122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1281262Z value_states = self.v(current_states) 2025-12-04T09:44:44.1281277Z 2025-12-04T09:44:44.1281413Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1281557Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1281739Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1282134Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1282249Z res = mod(**inputs) 2025-12-04T09:44:44.1282802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1282924Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1283456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1283602Z layer_outputs = layer_module( 2025-12-04T09:44:44.1284077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1284208Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1284773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1284903Z return func(*args, **kwargs) 2025-12-04T09:44:44.1285416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1285560Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1286083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1286197Z return func(*args, **kwargs) 2025-12-04T09:44:44.1286718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1286867Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1287383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1287505Z return func(*args, **kwargs) 2025-12-04T09:44:44.1288023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1288156Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1288171Z 2025-12-04T09:44:44.1288350Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1288753Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1288864Z res = mod(**inputs) 2025-12-04T09:44:44.1289385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1289511Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1290040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1290159Z layer_outputs = layer_module( 2025-12-04T09:44:44.1290638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1290769Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1291282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1291408Z return func(*args, **kwargs) 2025-12-04T09:44:44.1291923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1292076Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1292596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1292804Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1293324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1293489Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1293504Z 2025-12-04T09:44:44.1293686Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1294093Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1294233Z res = mod(**inputs) 2025-12-04T09:44:44.1294767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1294890Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1295408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1295567Z layer_outputs = layer_module( 2025-12-04T09:44:44.1296030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1296231Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1296752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1296872Z return func(*args, **kwargs) 2025-12-04T09:44:44.1297395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1297550Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1298061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1298277Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1298793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1298942Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1298957Z 2025-12-04T09:44:44.1299146Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1299546Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1299665Z res = mod(**inputs) 2025-12-04T09:44:44.1300181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1300306Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1301396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1301530Z layer_outputs = layer_module( 2025-12-04T09:44:44.1302035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1302180Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1302714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1302854Z return func(*args, **kwargs) 2025-12-04T09:44:44.1303385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1303547Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1304087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1304299Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1304844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1304993Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1305009Z 2025-12-04T09:44:44.1305195Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1305615Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1305730Z res = mod(**inputs) 2025-12-04T09:44:44.1306272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1306402Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1307014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1307235Z layer_outputs = layer_module( 2025-12-04T09:44:44.1307720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1307855Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1308444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1308562Z return func(*args, **kwargs) 2025-12-04T09:44:44.1309172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1309332Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1309862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1310084Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1310616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1310759Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1310775Z 2025-12-04T09:44:44.1310963Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1311383Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1311501Z res = mod(**inputs) 2025-12-04T09:44:44.1312041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1312175Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1312721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1312842Z layer_outputs = layer_module( 2025-12-04T09:44:44.1313335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1313478Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1314011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1314148Z return func(*args, **kwargs) 2025-12-04T09:44:44.1314689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1314839Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1315460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1315679Z return func(*args, **kwargs) 2025-12-04T09:44:44.1316235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1316385Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1316924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1317062Z return func(*args, **kwargs) 2025-12-04T09:44:44.1317600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1317756Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1317772Z 2025-12-04T09:44:44.1317965Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1318392Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1318522Z res = mod(**inputs) 2025-12-04T09:44:44.1319061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1319193Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1319748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1319916Z layer_outputs = layer_module( 2025-12-04T09:44:44.1320418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1320559Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1321141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1321277Z return func(*args, **kwargs) 2025-12-04T09:44:44.1321872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1322018Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1322568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1322692Z return func(*args, **kwargs) 2025-12-04T09:44:44.1323244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1323397Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1323930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1324060Z return func(*args, **kwargs) 2025-12-04T09:44:44.1324595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1324738Z key_states = self.k(current_states) 2025-12-04T09:44:44.1324759Z 2025-12-04T09:44:44.1324950Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1325366Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1325487Z res = mod(**inputs) 2025-12-04T09:44:44.1326026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1326159Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1326703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1326829Z layer_outputs = layer_module( 2025-12-04T09:44:44.1327439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1327578Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1328100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1328224Z return func(*args, **kwargs) 2025-12-04T09:44:44.1328735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1328869Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1329387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1329500Z return func(*args, **kwargs) 2025-12-04T09:44:44.1330024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1330165Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1330681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1330807Z return func(*args, **kwargs) 2025-12-04T09:44:44.1331325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1331464Z value_states = self.v(current_states) 2025-12-04T09:44:44.1331478Z 2025-12-04T09:44:44.1331611Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1331747Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1332007Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1332411Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1332519Z res = mod(**inputs) 2025-12-04T09:44:44.1333054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1333210Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1333742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1333914Z layer_outputs = layer_module( 2025-12-04T09:44:44.1334380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1334527Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1335044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1335170Z return func(*args, **kwargs) 2025-12-04T09:44:44.1335688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1335823Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1336357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1336476Z return func(*args, **kwargs) 2025-12-04T09:44:44.1336993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1337146Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1337658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1337780Z return func(*args, **kwargs) 2025-12-04T09:44:44.1338294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1338415Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1338430Z 2025-12-04T09:44:44.1338625Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1339031Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1339139Z res = mod(**inputs) 2025-12-04T09:44:44.1339668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1339794Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1340323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1340442Z layer_outputs = layer_module( 2025-12-04T09:44:44.1340905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1341051Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1341564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1341679Z return func(*args, **kwargs) 2025-12-04T09:44:44.1342211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1342348Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1342873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1342992Z return func(*args, **kwargs) 2025-12-04T09:44:44.1343503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1343655Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1344204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1344329Z return func(*args, **kwargs) 2025-12-04T09:44:44.1344843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1345015Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1345030Z 2025-12-04T09:44:44.1345224Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1345625Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1345783Z res = mod(**inputs) 2025-12-04T09:44:44.1346312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1346436Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1347084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1347381Z layer_outputs = layer_module( 2025-12-04T09:44:44.1347864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1348011Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1348544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1348663Z return func(*args, **kwargs) 2025-12-04T09:44:44.1349209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1349357Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1349897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1350019Z return func(*args, **kwargs) 2025-12-04T09:44:44.1350556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1350719Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1351249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1351379Z return func(*args, **kwargs) 2025-12-04T09:44:44.1351908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1352040Z key_states = self.k(current_states) 2025-12-04T09:44:44.1352056Z 2025-12-04T09:44:44.1352262Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1352681Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1352790Z res = mod(**inputs) 2025-12-04T09:44:44.1353337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1353469Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1354015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1354139Z layer_outputs = layer_module( 2025-12-04T09:44:44.1354630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1354778Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1355313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1355434Z return func(*args, **kwargs) 2025-12-04T09:44:44.1355972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1356113Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1356689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1356807Z return func(*args, **kwargs) 2025-12-04T09:44:44.1357333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1357523Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1358048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1358180Z return func(*args, **kwargs) 2025-12-04T09:44:44.1358760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1358892Z value_states = self.v(current_states) 2025-12-04T09:44:44.1358908Z 2025-12-04T09:44:44.1359060Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1359202Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1359506Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1359923Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1360028Z res = mod(**inputs) 2025-12-04T09:44:44.1360559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1360688Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1361210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1361349Z layer_outputs = layer_module( 2025-12-04T09:44:44.1361824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1361955Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1362480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1362595Z return func(*args, **kwargs) 2025-12-04T09:44:44.1363117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1363255Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1363770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1363900Z return func(*args, **kwargs) 2025-12-04T09:44:44.1364417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1364558Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1365081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1365195Z return func(*args, **kwargs) 2025-12-04T09:44:44.1365718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1365847Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1365861Z 2025-12-04T09:44:44.1366044Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1366459Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1366571Z res = mod(**inputs) 2025-12-04T09:44:44.1367102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1367227Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1367744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1367871Z layer_outputs = layer_module( 2025-12-04T09:44:44.1368338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1368505Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1369024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1369146Z return func(*args, **kwargs) 2025-12-04T09:44:44.1369673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1369857Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1370426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1370643Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1371159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1371333Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1371350Z 2025-12-04T09:44:44.1371534Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1371937Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1372048Z res = mod(**inputs) 2025-12-04T09:44:44.1372567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1372697Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1373223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1373346Z layer_outputs = layer_module( 2025-12-04T09:44:44.1373819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1373952Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1374468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1374597Z return func(*args, **kwargs) 2025-12-04T09:44:44.1375111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1375266Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1375788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1375996Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1376529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1376666Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1376681Z 2025-12-04T09:44:44.1376863Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1377275Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1377382Z res = mod(**inputs) 2025-12-04T09:44:44.1383193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1383326Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1383863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1384002Z layer_outputs = layer_module( 2025-12-04T09:44:44.1384490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1384627Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1385177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1385297Z return func(*args, **kwargs) 2025-12-04T09:44:44.1385840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1386077Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1386610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1387003Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1387540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1387702Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1387786Z 2025-12-04T09:44:44.1387980Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1388391Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1388522Z res = mod(**inputs) 2025-12-04T09:44:44.1389060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1389193Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1389743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1389870Z layer_outputs = layer_module( 2025-12-04T09:44:44.1390369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1390512Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1391048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1391179Z return func(*args, **kwargs) 2025-12-04T09:44:44.1391710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1391863Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1392405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1392619Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1393156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1393296Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1393312Z 2025-12-04T09:44:44.1393507Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1393929Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1394042Z res = mod(**inputs) 2025-12-04T09:44:44.1394583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1394708Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1395242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1395374Z layer_outputs = layer_module( 2025-12-04T09:44:44.1395857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1395994Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1396535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1396659Z return func(*args, **kwargs) 2025-12-04T09:44:44.1397207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1397355Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1397884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1398021Z return func(*args, **kwargs) 2025-12-04T09:44:44.1398595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1398849Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1399370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1399518Z return func(*args, **kwargs) 2025-12-04T09:44:44.1400041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1400221Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1400236Z 2025-12-04T09:44:44.1400425Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1401361Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1401481Z res = mod(**inputs) 2025-12-04T09:44:44.1402039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1402179Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1402715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1402851Z layer_outputs = layer_module( 2025-12-04T09:44:44.1403333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1403470Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1404016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1404132Z return func(*args, **kwargs) 2025-12-04T09:44:44.1404674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1404812Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1405336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1405463Z return func(*args, **kwargs) 2025-12-04T09:44:44.1405988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1406144Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1406672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1406791Z return func(*args, **kwargs) 2025-12-04T09:44:44.1407334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1407463Z key_states = self.k(current_states) 2025-12-04T09:44:44.1407478Z 2025-12-04T09:44:44.1407668Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1408088Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1408194Z res = mod(**inputs) 2025-12-04T09:44:44.1408742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1408870Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1409404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1409539Z layer_outputs = layer_module( 2025-12-04T09:44:44.1410026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1410161Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1410700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1410819Z return func(*args, **kwargs) 2025-12-04T09:44:44.1411452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1411594Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1412123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1412296Z return func(*args, **kwargs) 2025-12-04T09:44:44.1412831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1413060Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1413702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1413819Z return func(*args, **kwargs) 2025-12-04T09:44:44.1414340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1414471Z value_states = self.v(current_states) 2025-12-04T09:44:44.1414486Z 2025-12-04T09:44:44.1414618Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1414762Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1414947Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1415361Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1415464Z res = mod(**inputs) 2025-12-04T09:44:44.1415986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1416124Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1416646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1416768Z layer_outputs = layer_module( 2025-12-04T09:44:44.1417247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1417387Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1417917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1418032Z return func(*args, **kwargs) 2025-12-04T09:44:44.1418552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1418700Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1419220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1419336Z return func(*args, **kwargs) 2025-12-04T09:44:44.1419860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1420000Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1420522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1420637Z return func(*args, **kwargs) 2025-12-04T09:44:44.1421153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1421294Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1421309Z 2025-12-04T09:44:44.1421494Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1421910Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1422016Z res = mod(**inputs) 2025-12-04T09:44:44.1422533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1422669Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1423186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1423337Z layer_outputs = layer_module( 2025-12-04T09:44:44.1423817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1423952Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1424508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1424623Z return func(*args, **kwargs) 2025-12-04T09:44:44.1425193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1425343Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1425850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1425965Z return func(*args, **kwargs) 2025-12-04T09:44:44.1426488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 489, in forward 2025-12-04T09:44:44.1426796Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:44:44.1426821Z 2025-12-04T09:44:44.1427029Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1427605Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1427712Z res = mod(**inputs) 2025-12-04T09:44:44.1428263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1428390Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1428938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1429063Z layer_outputs = layer_module( 2025-12-04T09:44:44.1429547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1429704Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1430233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1430351Z return func(*args, **kwargs) 2025-12-04T09:44:44.1430897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1431042Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1431589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1431710Z return func(*args, **kwargs) 2025-12-04T09:44:44.1432240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1432402Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1432933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1433049Z return func(*args, **kwargs) 2025-12-04T09:44:44.1433588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1433720Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1433735Z 2025-12-04T09:44:44.1433929Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1434345Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1434455Z res = mod(**inputs) 2025-12-04T09:44:44.1435001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1435128Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1435673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1435835Z layer_outputs = layer_module( 2025-12-04T09:44:44.1436319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1436472Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1437034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1437154Z return func(*args, **kwargs) 2025-12-04T09:44:44.1437762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1437911Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1438448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1438567Z return func(*args, **kwargs) 2025-12-04T09:44:44.1439100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1439262Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1439790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1439924Z return func(*args, **kwargs) 2025-12-04T09:44:44.1445374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1445529Z key_states = self.k(current_states) 2025-12-04T09:44:44.1445545Z 2025-12-04T09:44:44.1445742Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1446157Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1446264Z res = mod(**inputs) 2025-12-04T09:44:44.1446809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1446942Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1447483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1447608Z layer_outputs = layer_module( 2025-12-04T09:44:44.1448096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1448239Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1448777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1448900Z return func(*args, **kwargs) 2025-12-04T09:44:44.1449438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1449586Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1450125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1450244Z return func(*args, **kwargs) 2025-12-04T09:44:44.1450778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1450945Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1451471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1451603Z return func(*args, **kwargs) 2025-12-04T09:44:44.1452250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1452377Z value_states = self.v(current_states) 2025-12-04T09:44:44.1452392Z 2025-12-04T09:44:44.1452546Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1452729Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1452917Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1453339Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1453448Z res = mod(**inputs) 2025-12-04T09:44:44.1453983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1454143Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1454668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1454860Z layer_outputs = layer_module( 2025-12-04T09:44:44.1455335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1455467Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1455993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1456112Z return func(*args, **kwargs) 2025-12-04T09:44:44.1456631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1456772Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1457289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1457414Z return func(*args, **kwargs) 2025-12-04T09:44:44.1457935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1458077Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1458608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1458728Z return func(*args, **kwargs) 2025-12-04T09:44:44.1459257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1459388Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1459402Z 2025-12-04T09:44:44.1459592Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1460011Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1460120Z res = mod(**inputs) 2025-12-04T09:44:44.1460657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1460787Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1461310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1461443Z layer_outputs = layer_module( 2025-12-04T09:44:44.1461909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1462046Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1462577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1462695Z return func(*args, **kwargs) 2025-12-04T09:44:44.1463229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1463383Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1463910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1464130Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1464646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1464830Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1464878Z 2025-12-04T09:44:44.1465065Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1465474Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1465600Z res = mod(**inputs) 2025-12-04T09:44:44.1466155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1466284Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1466970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1467099Z layer_outputs = layer_module( 2025-12-04T09:44:44.1467769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1467907Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1468445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1468586Z return func(*args, **kwargs) 2025-12-04T09:44:44.1469120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1469282Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1469826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1470041Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1470591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1470735Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1470751Z 2025-12-04T09:44:44.1470944Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1471379Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1471491Z res = mod(**inputs) 2025-12-04T09:44:44.1472038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1472170Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1472706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1472844Z layer_outputs = layer_module( 2025-12-04T09:44:44.1473328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1473465Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1474009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1474132Z return func(*args, **kwargs) 2025-12-04T09:44:44.1474673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1474832Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1475362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1475588Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1476120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1476283Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1476299Z 2025-12-04T09:44:44.1476490Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1476908Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1477024Z res = mod(**inputs) 2025-12-04T09:44:44.1477596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1477729Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1478275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1478430Z layer_outputs = layer_module( 2025-12-04T09:44:44.1478931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1479178Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1479747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1479879Z return func(*args, **kwargs) 2025-12-04T09:44:44.1480397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1480555Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1481077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1481280Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1481809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1481943Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1481958Z 2025-12-04T09:44:44.1482142Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1482595Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1482707Z res = mod(**inputs) 2025-12-04T09:44:44.1483240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1483366Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1483888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1484021Z layer_outputs = layer_module( 2025-12-04T09:44:44.1484488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1484625Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1485158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1485274Z return func(*args, **kwargs) 2025-12-04T09:44:44.1485803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1485942Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1486458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1486587Z return func(*args, **kwargs) 2025-12-04T09:44:44.1487103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1487246Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1487768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1487884Z return func(*args, **kwargs) 2025-12-04T09:44:44.1488411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1488541Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1488556Z 2025-12-04T09:44:44.1488738Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1489154Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1489261Z res = mod(**inputs) 2025-12-04T09:44:44.1490413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1490541Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1491060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1491230Z layer_outputs = layer_module( 2025-12-04T09:44:44.1491700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1491836Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1492433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1492550Z return func(*args, **kwargs) 2025-12-04T09:44:44.1493079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1493222Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1493734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1493862Z return func(*args, **kwargs) 2025-12-04T09:44:44.1494376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1494516Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1495045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1495165Z return func(*args, **kwargs) 2025-12-04T09:44:44.1495695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1495824Z key_states = self.k(current_states) 2025-12-04T09:44:44.1495838Z 2025-12-04T09:44:44.1496023Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1496437Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1496544Z res = mod(**inputs) 2025-12-04T09:44:44.1497074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1497199Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1497721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1497855Z layer_outputs = layer_module( 2025-12-04T09:44:44.1498325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1498459Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1498986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1499106Z return func(*args, **kwargs) 2025-12-04T09:44:44.1499630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1499769Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1500282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1500413Z return func(*args, **kwargs) 2025-12-04T09:44:44.1501521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1501700Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1502235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1502355Z return func(*args, **kwargs) 2025-12-04T09:44:44.1502967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1503295Z value_states = self.v(current_states) 2025-12-04T09:44:44.1503312Z 2025-12-04T09:44:44.1503457Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1503606Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1503799Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1504272Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1504382Z res = mod(**inputs) 2025-12-04T09:44:44.1504991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1505131Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1505667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1505791Z layer_outputs = layer_module( 2025-12-04T09:44:44.1506287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1506425Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1507092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1507216Z return func(*args, **kwargs) 2025-12-04T09:44:44.1507816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1507972Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1508509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1508631Z return func(*args, **kwargs) 2025-12-04T09:44:44.1509173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1509316Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1509854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1509974Z return func(*args, **kwargs) 2025-12-04T09:44:44.1510506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1510652Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1510668Z 2025-12-04T09:44:44.1510862Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1511296Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1511405Z res = mod(**inputs) 2025-12-04T09:44:44.1511944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1512086Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1512629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1512755Z layer_outputs = layer_module( 2025-12-04T09:44:44.1513252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1513394Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1513939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1514061Z return func(*args, **kwargs) 2025-12-04T09:44:44.1514596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1514753Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1515280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1515440Z return func(*args, **kwargs) 2025-12-04T09:44:44.1515985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1516135Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1516681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1516836Z return func(*args, **kwargs) 2025-12-04T09:44:44.1517372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1517579Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1517595Z 2025-12-04T09:44:44.1517789Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1518221Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1518333Z res = mod(**inputs) 2025-12-04T09:44:44.1518873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1519020Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1519667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1519795Z layer_outputs = layer_module( 2025-12-04T09:44:44.1520283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1520421Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1520954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1521074Z return func(*args, **kwargs) 2025-12-04T09:44:44.1521591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1521748Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1522261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1522376Z return func(*args, **kwargs) 2025-12-04T09:44:44.1522904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1523056Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1523582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1523704Z return func(*args, **kwargs) 2025-12-04T09:44:44.1524224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1524366Z key_states = self.k(current_states) 2025-12-04T09:44:44.1524381Z 2025-12-04T09:44:44.1524567Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1524988Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1525097Z res = mod(**inputs) 2025-12-04T09:44:44.1525626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1525770Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1526294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1526417Z layer_outputs = layer_module( 2025-12-04T09:44:44.1526900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1527035Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1527569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1527722Z return func(*args, **kwargs) 2025-12-04T09:44:44.1528236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1528393Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1528913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1529059Z return func(*args, **kwargs) 2025-12-04T09:44:44.1529585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1529809Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1530340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1530459Z return func(*args, **kwargs) 2025-12-04T09:44:44.1530972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1531116Z value_states = self.v(current_states) 2025-12-04T09:44:44.1531131Z 2025-12-04T09:44:44.1531268Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1531416Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1531602Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1532012Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1532132Z res = mod(**inputs) 2025-12-04T09:44:44.1532654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1532778Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1533309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1533428Z layer_outputs = layer_module( 2025-12-04T09:44:44.1533911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1534050Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1534567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1534700Z return func(*args, **kwargs) 2025-12-04T09:44:44.1535219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1535360Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1535893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1536012Z return func(*args, **kwargs) 2025-12-04T09:44:44.1536538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1536682Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1537195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1537322Z return func(*args, **kwargs) 2025-12-04T09:44:44.1537836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1537964Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1537991Z 2025-12-04T09:44:44.1538176Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1538582Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1538696Z res = mod(**inputs) 2025-12-04T09:44:44.1539216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1539339Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1539869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1540023Z layer_outputs = layer_module( 2025-12-04T09:44:44.1540501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1540636Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1541195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1541324Z return func(*args, **kwargs) 2025-12-04T09:44:44.1541895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1542037Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1542563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1542683Z return func(*args, **kwargs) 2025-12-04T09:44:44.1543208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 529, in forward 2025-12-04T09:44:44.1543443Z layer_output = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:44:44.1543459Z 2025-12-04T09:44:44.1543647Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1544062Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1544170Z res = mod(**inputs) 2025-12-04T09:44:44.1544710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1544836Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1545358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1545495Z layer_outputs = layer_module( 2025-12-04T09:44:44.1545964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1546100Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1546630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1546852Z return func(*args, **kwargs) 2025-12-04T09:44:44.1547569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1547729Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1548270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1548499Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1549032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1549208Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1549238Z 2025-12-04T09:44:44.1549428Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1549846Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1549976Z res = mod(**inputs) 2025-12-04T09:44:44.1550514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1550645Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1551199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1551325Z layer_outputs = layer_module( 2025-12-04T09:44:44.1551823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1551960Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1552533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1552670Z return func(*args, **kwargs) 2025-12-04T09:44:44.1553203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1553392Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1553939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1554205Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1554752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1554894Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1554910Z 2025-12-04T09:44:44.1555098Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1555535Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1555643Z res = mod(**inputs) 2025-12-04T09:44:44.1556198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1556331Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1556870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1557009Z layer_outputs = layer_module( 2025-12-04T09:44:44.1557497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1557636Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1558178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1558301Z return func(*args, **kwargs) 2025-12-04T09:44:44.1558845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1559004Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1559626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1559848Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1560365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1560510Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1560539Z 2025-12-04T09:44:44.1560727Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1561129Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1561250Z res = mod(**inputs) 2025-12-04T09:44:44.1561772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1561899Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1562430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1562554Z layer_outputs = layer_module( 2025-12-04T09:44:44.1563036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1563176Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1563697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1563830Z return func(*args, **kwargs) 2025-12-04T09:44:44.1564347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1564537Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1577389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1577678Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1578331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1578472Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1578489Z 2025-12-04T09:44:44.1578752Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1579288Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1579408Z res = mod(**inputs) 2025-12-04T09:44:44.1579938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1580072Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1580606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1580729Z layer_outputs = layer_module( 2025-12-04T09:44:44.1581213Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1581355Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1581876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1582019Z return func(*args, **kwargs) 2025-12-04T09:44:44.1582538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1582678Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1583207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1583329Z return func(*args, **kwargs) 2025-12-04T09:44:44.1583864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1584008Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1584531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1584663Z return func(*args, **kwargs) 2025-12-04T09:44:44.1585179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1585312Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1585327Z 2025-12-04T09:44:44.1585514Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1585918Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1586035Z res = mod(**inputs) 2025-12-04T09:44:44.1586558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1586680Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1587523Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1587653Z layer_outputs = layer_module( 2025-12-04T09:44:44.1588148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1588291Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1588823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1588956Z return func(*args, **kwargs) 2025-12-04T09:44:44.1589488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1589666Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1590212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1590330Z return func(*args, **kwargs) 2025-12-04T09:44:44.1590904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1591050Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1591630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1591760Z return func(*args, **kwargs) 2025-12-04T09:44:44.1592293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1592433Z key_states = self.k(current_states) 2025-12-04T09:44:44.1592452Z 2025-12-04T09:44:44.1592646Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1593061Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1593182Z res = mod(**inputs) 2025-12-04T09:44:44.1593724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1593858Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1594398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1594529Z layer_outputs = layer_module( 2025-12-04T09:44:44.1595023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1595165Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1595696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1595828Z return func(*args, **kwargs) 2025-12-04T09:44:44.1596359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1596499Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1597042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1597161Z return func(*args, **kwargs) 2025-12-04T09:44:44.1597708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1597852Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1598382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1598513Z return func(*args, **kwargs) 2025-12-04T09:44:44.1599154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1599301Z value_states = self.v(current_states) 2025-12-04T09:44:44.1599316Z 2025-12-04T09:44:44.1599453Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1599586Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1599786Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1600188Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1600295Z res = mod(**inputs) 2025-12-04T09:44:44.1601435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1601576Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1602133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1602264Z layer_outputs = layer_module( 2025-12-04T09:44:44.1602873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1603026Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1603558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1603723Z return func(*args, **kwargs) 2025-12-04T09:44:44.1604264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1604483Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1605028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1605149Z return func(*args, **kwargs) 2025-12-04T09:44:44.1605672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1605829Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1606356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1606476Z return func(*args, **kwargs) 2025-12-04T09:44:44.1607020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1607155Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1607172Z 2025-12-04T09:44:44.1607376Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1607792Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1607901Z res = mod(**inputs) 2025-12-04T09:44:44.1608451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1608579Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1609127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1609252Z layer_outputs = layer_module( 2025-12-04T09:44:44.1609735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1609888Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1610418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1610539Z return func(*args, **kwargs) 2025-12-04T09:44:44.1611086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1611232Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1611772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1611894Z return func(*args, **kwargs) 2025-12-04T09:44:44.1612422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1612586Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1613113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1613250Z return func(*args, **kwargs) 2025-12-04T09:44:44.1613893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1614020Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1614035Z 2025-12-04T09:44:44.1614231Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1614634Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1614770Z res = mod(**inputs) 2025-12-04T09:44:44.1615300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1615427Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1615951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1616102Z layer_outputs = layer_module( 2025-12-04T09:44:44.1616570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1616768Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1617283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1617396Z return func(*args, **kwargs) 2025-12-04T09:44:44.1617919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1618064Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1618581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1618695Z return func(*args, **kwargs) 2025-12-04T09:44:44.1619205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1619360Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1619874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1620002Z return func(*args, **kwargs) 2025-12-04T09:44:44.1620511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1620637Z key_states = self.k(current_states) 2025-12-04T09:44:44.1620652Z 2025-12-04T09:44:44.1620847Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1621250Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1621360Z res = mod(**inputs) 2025-12-04T09:44:44.1621890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1622018Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1622551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1622673Z layer_outputs = layer_module( 2025-12-04T09:44:44.1623141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1623287Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1623801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1623921Z return func(*args, **kwargs) 2025-12-04T09:44:44.1624450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1624589Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1625116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1625235Z return func(*args, **kwargs) 2025-12-04T09:44:44.1625760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1625924Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1626436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1626558Z return func(*args, **kwargs) 2025-12-04T09:44:44.1627345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1627556Z value_states = self.v(current_states) 2025-12-04T09:44:44.1627574Z 2025-12-04T09:44:44.1627791Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1627936Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1633132Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1633568Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1633680Z res = mod(**inputs) 2025-12-04T09:44:44.1634302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1634434Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1634969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1635107Z layer_outputs = layer_module( 2025-12-04T09:44:44.1635594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1635729Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1636273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1636395Z return func(*args, **kwargs) 2025-12-04T09:44:44.1636938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1637086Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1637616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1637747Z return func(*args, **kwargs) 2025-12-04T09:44:44.1638278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1638429Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1638972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1639092Z return func(*args, **kwargs) 2025-12-04T09:44:44.1639633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1639763Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1639779Z 2025-12-04T09:44:44.1639968Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1640499Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1640609Z res = mod(**inputs) 2025-12-04T09:44:44.1641136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1641261Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1641776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1641908Z layer_outputs = layer_module( 2025-12-04T09:44:44.1642375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1642511Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1643042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1643163Z return func(*args, **kwargs) 2025-12-04T09:44:44.1643695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1643852Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1644369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1644620Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1645133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1645309Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1645354Z 2025-12-04T09:44:44.1645535Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1645934Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1646052Z res = mod(**inputs) 2025-12-04T09:44:44.1646638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1646763Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1647284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1647405Z layer_outputs = layer_module( 2025-12-04T09:44:44.1647887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1648021Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1648533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1648664Z return func(*args, **kwargs) 2025-12-04T09:44:44.1649174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1649333Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1649860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1650065Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1650590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1650729Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1650744Z 2025-12-04T09:44:44.1650928Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1651339Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1651449Z res = mod(**inputs) 2025-12-04T09:44:44.1651980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1652103Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1652628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1652761Z layer_outputs = layer_module( 2025-12-04T09:44:44.1653228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1653361Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1653887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1654004Z return func(*args, **kwargs) 2025-12-04T09:44:44.1654527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1654682Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1655201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1655413Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1655930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1656090Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1656135Z 2025-12-04T09:44:44.1656321Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1656716Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1656832Z res = mod(**inputs) 2025-12-04T09:44:44.1657353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1657508Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1658037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1658214Z layer_outputs = layer_module( 2025-12-04T09:44:44.1658697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1658832Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1659347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1659480Z return func(*args, **kwargs) 2025-12-04T09:44:44.1659995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1660146Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1660673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1660879Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1661409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1661541Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1661556Z 2025-12-04T09:44:44.1661740Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1662147Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1662256Z res = mod(**inputs) 2025-12-04T09:44:44.1662784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1662911Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1663430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1663562Z layer_outputs = layer_module( 2025-12-04T09:44:44.1664031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1664169Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1664699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1664818Z return func(*args, **kwargs) 2025-12-04T09:44:44.1665340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1665493Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1666003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 218, in forward 2025-12-04T09:44:44.1666239Z hidden_states = hidden_states + self.dropout(forwarded_states) 2025-12-04T09:44:44.1666257Z 2025-12-04T09:44:44.1666442Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1666969Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1667083Z res = mod(**inputs) 2025-12-04T09:44:44.1667798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1667926Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1668460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1668634Z layer_outputs = layer_module( 2025-12-04T09:44:44.1669112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1669253Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1669795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1669948Z return func(*args, **kwargs) 2025-12-04T09:44:44.1670543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1670688Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1671215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1671345Z return func(*args, **kwargs) 2025-12-04T09:44:44.1671878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1672030Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1672565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1672688Z return func(*args, **kwargs) 2025-12-04T09:44:44.1673224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1673356Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1673371Z 2025-12-04T09:44:44.1673567Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1673987Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1674098Z res = mod(**inputs) 2025-12-04T09:44:44.1674641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1674772Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1675306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1675437Z layer_outputs = layer_module( 2025-12-04T09:44:44.1675921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1676062Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1676603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1676722Z return func(*args, **kwargs) 2025-12-04T09:44:44.1677264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1677406Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1677936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1678069Z return func(*args, **kwargs) 2025-12-04T09:44:44.1678600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1678748Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1679394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1679511Z return func(*args, **kwargs) 2025-12-04T09:44:44.1680040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1680169Z key_states = self.k(current_states) 2025-12-04T09:44:44.1680184Z 2025-12-04T09:44:44.1680363Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1680773Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1680904Z res = mod(**inputs) 2025-12-04T09:44:44.1681431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1681549Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1682060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1682215Z layer_outputs = layer_module( 2025-12-04T09:44:44.1682729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1682863Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1683377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1683490Z return func(*args, **kwargs) 2025-12-04T09:44:44.1684002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1684138Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1684649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1684772Z return func(*args, **kwargs) 2025-12-04T09:44:44.1685286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1685422Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1685943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1686057Z return func(*args, **kwargs) 2025-12-04T09:44:44.1686578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1686704Z value_states = self.v(current_states) 2025-12-04T09:44:44.1686722Z 2025-12-04T09:44:44.1686854Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1686995Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1687175Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1687582Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1687689Z res = mod(**inputs) 2025-12-04T09:44:44.1688206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1688334Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1688851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1688966Z layer_outputs = layer_module( 2025-12-04T09:44:44.1689439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1689575Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1690162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1690276Z return func(*args, **kwargs) 2025-12-04T09:44:44.1695784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1695941Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1696468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1696593Z return func(*args, **kwargs) 2025-12-04T09:44:44.1697133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1697274Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1697818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1697984Z return func(*args, **kwargs) 2025-12-04T09:44:44.1698508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1698649Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1698696Z 2025-12-04T09:44:44.1698886Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1699313Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1699423Z res = mod(**inputs) 2025-12-04T09:44:44.1700028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1700168Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1700703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1701220Z layer_outputs = layer_module( 2025-12-04T09:44:44.1701723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1701859Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1702408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1702532Z return func(*args, **kwargs) 2025-12-04T09:44:44.1703060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1703220Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1703750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1703869Z return func(*args, **kwargs) 2025-12-04T09:44:44.1704404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1704551Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1705085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1705200Z return func(*args, **kwargs) 2025-12-04T09:44:44.1705726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1705862Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1705878Z 2025-12-04T09:44:44.1706070Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1706486Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1706593Z res = mod(**inputs) 2025-12-04T09:44:44.1707224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1707363Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1707894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1708015Z layer_outputs = layer_module( 2025-12-04T09:44:44.1708498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1708634Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1709165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1709283Z return func(*args, **kwargs) 2025-12-04T09:44:44.1709812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1709964Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1710489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1710687Z return func(*args, **kwargs) 2025-12-04T09:44:44.1711229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1711377Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1711950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1712065Z return func(*args, **kwargs) 2025-12-04T09:44:44.1712666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1712804Z key_states = self.k(current_states) 2025-12-04T09:44:44.1712820Z 2025-12-04T09:44:44.1713007Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1713427Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1713536Z res = mod(**inputs) 2025-12-04T09:44:44.1714071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1714206Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1714744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1714873Z layer_outputs = layer_module( 2025-12-04T09:44:44.1715360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1715500Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1716039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1716154Z return func(*args, **kwargs) 2025-12-04T09:44:44.1716681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1716834Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1717361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1717483Z return func(*args, **kwargs) 2025-12-04T09:44:44.1718024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1718174Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1718820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1718931Z return func(*args, **kwargs) 2025-12-04T09:44:44.1719443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1719579Z value_states = self.v(current_states) 2025-12-04T09:44:44.1719596Z 2025-12-04T09:44:44.1719730Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1719870Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1720049Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1720447Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1720564Z res = mod(**inputs) 2025-12-04T09:44:44.1721082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1721205Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1721732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1721851Z layer_outputs = layer_module( 2025-12-04T09:44:44.1722321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1722481Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1722995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1723116Z return func(*args, **kwargs) 2025-12-04T09:44:44.1723625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1723790Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1724309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1724479Z return func(*args, **kwargs) 2025-12-04T09:44:44.1724996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1725135Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1725644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1725766Z return func(*args, **kwargs) 2025-12-04T09:44:44.1726276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1726401Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1726429Z 2025-12-04T09:44:44.1726609Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1727004Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1727114Z res = mod(**inputs) 2025-12-04T09:44:44.1727632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1727758Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1728281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1728400Z layer_outputs = layer_module( 2025-12-04T09:44:44.1728873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1729002Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1729512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1729637Z return func(*args, **kwargs) 2025-12-04T09:44:44.1730145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1730300Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1730820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1731020Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1731542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1731708Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1731722Z 2025-12-04T09:44:44.1731898Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1732300Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1732409Z res = mod(**inputs) 2025-12-04T09:44:44.1732933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1733057Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1733574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1733701Z layer_outputs = layer_module( 2025-12-04T09:44:44.1734162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1734322Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1734840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1734951Z return func(*args, **kwargs) 2025-12-04T09:44:44.1735474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1735651Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1736216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1736427Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1736935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1737067Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1737096Z 2025-12-04T09:44:44.1737276Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1737672Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1737784Z res = mod(**inputs) 2025-12-04T09:44:44.1738303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1738429Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1738952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1739071Z layer_outputs = layer_module( 2025-12-04T09:44:44.1739543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1739670Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1740181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1740305Z return func(*args, **kwargs) 2025-12-04T09:44:44.1740818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1740968Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1741494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1741692Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1742220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1742360Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1742374Z 2025-12-04T09:44:44.1742558Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1742968Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1743077Z res = mod(**inputs) 2025-12-04T09:44:44.1743609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1743730Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1744247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1744377Z layer_outputs = layer_module( 2025-12-04T09:44:44.1744841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1744970Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1745489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1745604Z return func(*args, **kwargs) 2025-12-04T09:44:44.1746123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1746303Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1746938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1747376Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1747905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1748039Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1748151Z 2025-12-04T09:44:44.1748339Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1748748Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1748871Z res = mod(**inputs) 2025-12-04T09:44:44.1749403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1749534Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1750076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1750199Z layer_outputs = layer_module( 2025-12-04T09:44:44.1750696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1750835Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1751366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1751500Z return func(*args, **kwargs) 2025-12-04T09:44:44.1752028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1752165Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1752772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1752888Z return func(*args, **kwargs) 2025-12-04T09:44:44.1753537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1753683Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1754212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1754342Z return func(*args, **kwargs) 2025-12-04T09:44:44.1754877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1755008Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1755035Z 2025-12-04T09:44:44.1755223Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1755634Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1755756Z res = mod(**inputs) 2025-12-04T09:44:44.1756287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1756413Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1756960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1757077Z layer_outputs = layer_module( 2025-12-04T09:44:44.1757638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1757772Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1758302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1758428Z return func(*args, **kwargs) 2025-12-04T09:44:44.1759002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1759140Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1759676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1759825Z return func(*args, **kwargs) 2025-12-04T09:44:44.1760369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1760509Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1761089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1761217Z return func(*args, **kwargs) 2025-12-04T09:44:44.1761743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1761877Z key_states = self.k(current_states) 2025-12-04T09:44:44.1761905Z 2025-12-04T09:44:44.1762094Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1762504Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1762625Z res = mod(**inputs) 2025-12-04T09:44:44.1763157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1763283Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1763821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1763938Z layer_outputs = layer_module( 2025-12-04T09:44:44.1764421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1764666Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1765177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1765301Z return func(*args, **kwargs) 2025-12-04T09:44:44.1765810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1765945Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1766467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1766580Z return func(*args, **kwargs) 2025-12-04T09:44:44.1767105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1767241Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1767747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1767871Z return func(*args, **kwargs) 2025-12-04T09:44:44.1768377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1768508Z value_states = self.v(current_states) 2025-12-04T09:44:44.1768524Z 2025-12-04T09:44:44.1768654Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1768788Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1768977Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1769371Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1769480Z res = mod(**inputs) 2025-12-04T09:44:44.1770009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1770127Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1770649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1770795Z layer_outputs = layer_module( 2025-12-04T09:44:44.1771259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1771402Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1771913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1772056Z return func(*args, **kwargs) 2025-12-04T09:44:44.1772632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1772768Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1773291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1773403Z return func(*args, **kwargs) 2025-12-04T09:44:44.1773916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1774068Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1774574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1774689Z return func(*args, **kwargs) 2025-12-04T09:44:44.1775208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1775335Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1775350Z 2025-12-04T09:44:44.1775547Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1775946Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1776052Z res = mod(**inputs) 2025-12-04T09:44:44.1776576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1776701Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1777219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1777336Z layer_outputs = layer_module( 2025-12-04T09:44:44.1777799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1777942Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1778458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1778574Z return func(*args, **kwargs) 2025-12-04T09:44:44.1779092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1779224Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1779742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1779854Z return func(*args, **kwargs) 2025-12-04T09:44:44.1780368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 489, in forward 2025-12-04T09:44:44.1780610Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:44:44.1780627Z 2025-12-04T09:44:44.1780806Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1781213Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1781321Z res = mod(**inputs) 2025-12-04T09:44:44.1781834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1781971Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1782485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1782636Z layer_outputs = layer_module( 2025-12-04T09:44:44.1783107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1783237Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1783786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1783899Z return func(*args, **kwargs) 2025-12-04T09:44:44.1784463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1784608Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1785117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1785229Z return func(*args, **kwargs) 2025-12-04T09:44:44.1785748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1785891Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1786412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1786525Z return func(*args, **kwargs) 2025-12-04T09:44:44.1787299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1787447Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1787472Z 2025-12-04T09:44:44.1787665Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1788084Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1788193Z res = mod(**inputs) 2025-12-04T09:44:44.1788726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1788865Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1789399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1789518Z layer_outputs = layer_module( 2025-12-04T09:44:44.1790010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1790144Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1790682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1790797Z return func(*args, **kwargs) 2025-12-04T09:44:44.1791324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1791474Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1792000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1792117Z return func(*args, **kwargs) 2025-12-04T09:44:44.1792651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1792800Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1793337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1793450Z return func(*args, **kwargs) 2025-12-04T09:44:44.1793977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1794114Z key_states = self.k(current_states) 2025-12-04T09:44:44.1794129Z 2025-12-04T09:44:44.1794320Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1794750Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1794914Z res = mod(**inputs) 2025-12-04T09:44:44.1795452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1795602Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1796172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1796301Z layer_outputs = layer_module( 2025-12-04T09:44:44.1796859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1797002Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1797549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1797672Z return func(*args, **kwargs) 2025-12-04T09:44:44.1798206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1798366Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1798896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1799019Z return func(*args, **kwargs) 2025-12-04T09:44:44.1799662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1799808Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1800339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1800456Z return func(*args, **kwargs) 2025-12-04T09:44:44.1801527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1801697Z value_states = self.v(current_states) 2025-12-04T09:44:44.1801713Z 2025-12-04T09:44:44.1801860Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1802030Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1802226Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1802642Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1802778Z res = mod(**inputs) 2025-12-04T09:44:44.1803319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1803448Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1803994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1804120Z layer_outputs = layer_module( 2025-12-04T09:44:44.1804608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1804749Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1805279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1805408Z return func(*args, **kwargs) 2025-12-04T09:44:44.1805939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1806087Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1806629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1806750Z return func(*args, **kwargs) 2025-12-04T09:44:44.1807292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1807444Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1807976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1808194Z return func(*args, **kwargs) 2025-12-04T09:44:44.1808726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1808898Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1808921Z 2025-12-04T09:44:44.1809107Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1809521Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1809721Z res = mod(**inputs) 2025-12-04T09:44:44.1810262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1810389Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1810934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1811063Z layer_outputs = layer_module( 2025-12-04T09:44:44.1811551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1811689Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1812224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1812358Z return func(*args, **kwargs) 2025-12-04T09:44:44.1812896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1813055Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1813599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1813816Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1814358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1814532Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1814547Z 2025-12-04T09:44:44.1814739Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1815230Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1815348Z res = mod(**inputs) 2025-12-04T09:44:44.1815990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1816123Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1816657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1816795Z layer_outputs = layer_module( 2025-12-04T09:44:44.1817272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1817412Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1817952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1818073Z return func(*args, **kwargs) 2025-12-04T09:44:44.1818615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1818769Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1819303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1819532Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1820066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1820204Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1820269Z 2025-12-04T09:44:44.1820460Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1820873Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1820991Z res = mod(**inputs) 2025-12-04T09:44:44.1821557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1821684Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1822286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1822414Z layer_outputs = layer_module( 2025-12-04T09:44:44.1822907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1823042Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1823572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1823702Z return func(*args, **kwargs) 2025-12-04T09:44:44.1824234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1824389Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1824930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1825142Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1825686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1825837Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1825852Z 2025-12-04T09:44:44.1826043Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1826467Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1826574Z res = mod(**inputs) 2025-12-04T09:44:44.1827228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1827356Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1827894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1828025Z layer_outputs = layer_module( 2025-12-04T09:44:44.1828514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1828654Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1829197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1829319Z return func(*args, **kwargs) 2025-12-04T09:44:44.1829864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1830022Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1830553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1830781Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1831315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1831456Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1831489Z 2025-12-04T09:44:44.1831676Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1832092Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1832209Z res = mod(**inputs) 2025-12-04T09:44:44.1832750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1832918Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1833464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1833623Z layer_outputs = layer_module( 2025-12-04T09:44:44.1834112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1834250Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1834848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1834979Z return func(*args, **kwargs) 2025-12-04T09:44:44.1835509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1835651Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1836191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1836311Z return func(*args, **kwargs) 2025-12-04T09:44:44.1836846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1836992Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1837520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1837653Z return func(*args, **kwargs) 2025-12-04T09:44:44.1838180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1838309Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1838338Z 2025-12-04T09:44:44.1838527Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1839047Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1839163Z res = mod(**inputs) 2025-12-04T09:44:44.1839678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1839805Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1840334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1840454Z layer_outputs = layer_module( 2025-12-04T09:44:44.1840936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1841070Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1841583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1841705Z return func(*args, **kwargs) 2025-12-04T09:44:44.1842220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1842360Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1842883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1842999Z return func(*args, **kwargs) 2025-12-04T09:44:44.1843521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1843663Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1844181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1844301Z return func(*args, **kwargs) 2025-12-04T09:44:44.1844812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1844970Z key_states = self.k(current_states) 2025-12-04T09:44:44.1844995Z 2025-12-04T09:44:44.1845177Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1845577Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1845725Z res = mod(**inputs) 2025-12-04T09:44:44.1846244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1846367Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1846951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1847072Z layer_outputs = layer_module( 2025-12-04T09:44:44.1847550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1847684Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1848195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1848320Z return func(*args, **kwargs) 2025-12-04T09:44:44.1848832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1848972Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1849496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1849619Z return func(*args, **kwargs) 2025-12-04T09:44:44.1850143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1850285Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1850799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1850928Z return func(*args, **kwargs) 2025-12-04T09:44:44.1851440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1851567Z value_states = self.v(current_states) 2025-12-04T09:44:44.1851595Z 2025-12-04T09:44:44.1851735Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1851869Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1852065Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1852469Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1852577Z res = mod(**inputs) 2025-12-04T09:44:44.1853102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1853227Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1853761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1853880Z layer_outputs = layer_module( 2025-12-04T09:44:44.1854343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1854483Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1854999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1855118Z return func(*args, **kwargs) 2025-12-04T09:44:44.1855647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 565, in forward 2025-12-04T09:44:44.1855787Z self_attention_outputs = self.layer[0]( 2025-12-04T09:44:44.1856306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1856423Z return func(*args, **kwargs) 2025-12-04T09:44:44.1856968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 479, in forward 2025-12-04T09:44:44.1857121Z attention_output = self.SelfAttention( 2025-12-04T09:44:44.1857634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1857780Z return func(*args, **kwargs) 2025-12-04T09:44:44.1858303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1858429Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1858498Z 2025-12-04T09:44:44.1858696Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1859096Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1859202Z res = mod(**inputs) 2025-12-04T09:44:44.1859732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1859861Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1860393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1860511Z layer_outputs = layer_module( 2025-12-04T09:44:44.1860982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1861126Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1861646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1861764Z return func(*args, **kwargs) 2025-12-04T09:44:44.1862283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1862424Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1862943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1863057Z return func(*args, **kwargs) 2025-12-04T09:44:44.1863574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1863740Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1864256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1864371Z return func(*args, **kwargs) 2025-12-04T09:44:44.1864908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 367, in forward 2025-12-04T09:44:44.1865041Z query_states = self.q(hidden_states) 2025-12-04T09:44:44.1865056Z 2025-12-04T09:44:44.1865253Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1865659Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1865764Z res = mod(**inputs) 2025-12-04T09:44:44.1866293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1866418Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1867041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1867336Z layer_outputs = layer_module( 2025-12-04T09:44:44.1867824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1867972Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1868507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1868626Z return func(*args, **kwargs) 2025-12-04T09:44:44.1869209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1869353Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1869886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1870039Z return func(*args, **kwargs) 2025-12-04T09:44:44.1870570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1870725Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1871314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1871434Z return func(*args, **kwargs) 2025-12-04T09:44:44.1871974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 388, in forward 2025-12-04T09:44:44.1872114Z key_states = self.k(current_states) 2025-12-04T09:44:44.1872129Z 2025-12-04T09:44:44.1872334Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1872754Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1872867Z res = mod(**inputs) 2025-12-04T09:44:44.1873414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1873543Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1874095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1874223Z layer_outputs = layer_module( 2025-12-04T09:44:44.1874709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1874852Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1875391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1875508Z return func(*args, **kwargs) 2025-12-04T09:44:44.1876047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1876196Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1876741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1876863Z return func(*args, **kwargs) 2025-12-04T09:44:44.1877403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1877621Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1883127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1883249Z return func(*args, **kwargs) 2025-12-04T09:44:44.1883793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 389, in forward 2025-12-04T09:44:44.1883926Z value_states = self.v(current_states) 2025-12-04T09:44:44.1883942Z 2025-12-04T09:44:44.1884092Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1884233Z cudagraph partition due to non gpu ops 2025-12-04T09:44:44.1884422Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1884844Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1884958Z res = mod(**inputs) 2025-12-04T09:44:44.1885492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1885629Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1886158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1886340Z layer_outputs = layer_module( 2025-12-04T09:44:44.1886818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1886955Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1887528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1887647Z return func(*args, **kwargs) 2025-12-04T09:44:44.1888259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1888405Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1888938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1889067Z return func(*args, **kwargs) 2025-12-04T09:44:44.1889602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 517, in forward 2025-12-04T09:44:44.1889746Z attention_output = self.EncDecAttention( 2025-12-04T09:44:44.1890384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1890505Z return func(*args, **kwargs) 2025-12-04T09:44:44.1891031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 447, in forward 2025-12-04T09:44:44.1891163Z attn_output = self.o(attn_output) 2025-12-04T09:44:44.1891178Z 2025-12-04T09:44:44.1891367Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1891788Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1891899Z res = mod(**inputs) 2025-12-04T09:44:44.1892431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1892560Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1893080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1893207Z layer_outputs = layer_module( 2025-12-04T09:44:44.1893681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1893813Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1894343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1894462Z return func(*args, **kwargs) 2025-12-04T09:44:44.1894988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 589, in forward 2025-12-04T09:44:44.1895129Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:44:44.1895647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1895775Z return func(*args, **kwargs) 2025-12-04T09:44:44.1896294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 529, in forward 2025-12-04T09:44:44.1896528Z layer_output = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:44:44.1896555Z 2025-12-04T09:44:44.1896743Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1897150Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1897269Z res = mod(**inputs) 2025-12-04T09:44:44.1897793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1897922Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1898455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1898612Z layer_outputs = layer_module( 2025-12-04T09:44:44.1899097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1899231Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1899784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1899913Z return func(*args, **kwargs) 2025-12-04T09:44:44.1900483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1900640Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1901718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1901936Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1902483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 184, in forward 2025-12-04T09:44:44.1902654Z hidden_gelu = self.act(self.wi_0(hidden_states)) 2025-12-04T09:44:44.1902670Z 2025-12-04T09:44:44.1902865Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1903299Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1903412Z res = mod(**inputs) 2025-12-04T09:44:44.1903971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1904104Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1904643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1904783Z layer_outputs = layer_module( 2025-12-04T09:44:44.1905266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1905405Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1905949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1906072Z return func(*args, **kwargs) 2025-12-04T09:44:44.1906615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1906882Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1907425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1907650Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1908184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 185, in forward 2025-12-04T09:44:44.1908330Z hidden_linear = self.wi_1(hidden_states) 2025-12-04T09:44:44.1908359Z 2025-12-04T09:44:44.1908550Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1908967Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1909095Z res = mod(**inputs) 2025-12-04T09:44:44.1909635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1909767Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1910325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1910452Z layer_outputs = layer_module( 2025-12-04T09:44:44.1910944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1911083Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1911703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1911842Z return func(*args, **kwargs) 2025-12-04T09:44:44.1912378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1912580Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1913128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1913415Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1913959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 186, in forward 2025-12-04T09:44:44.1914112Z hidden_states = hidden_gelu * hidden_linear 2025-12-04T09:44:44.1914129Z 2025-12-04T09:44:44.1914319Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1914745Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1914859Z res = mod(**inputs) 2025-12-04T09:44:44.1915405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1787, in forward 2025-12-04T09:44:44.1915538Z decoder_outputs = self.decoder( 2025-12-04T09:44:44.1916074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1087, in forward 2025-12-04T09:44:44.1916214Z layer_outputs = layer_module( 2025-12-04T09:44:44.1916702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:44:44.1916844Z return super().__call__(*args, **kwargs) 2025-12-04T09:44:44.1917394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:44:44.1917523Z return func(*args, **kwargs) 2025-12-04T09:44:44.1918069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 615, in forward 2025-12-04T09:44:44.1918230Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:44:44.1918872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 217, in forward 2025-12-04T09:44:44.1919096Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:44:44.1919614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 199, in forward 2025-12-04T09:44:44.1919752Z hidden_states = self.wo(hidden_states) 2025-12-04T09:44:44.1919782Z 2025-12-04T09:44:44.1919969Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1920374Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1920488Z res = mod(**inputs) 2025-12-04T09:44:44.1921015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1816, in forward 2025-12-04T09:44:44.1921161Z lm_logits = self.lm_head(sequence_output) 2025-12-04T09:44:44.1921177Z 2025-12-04T09:44:44.1921372Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:44:44.1921773Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:44:44.1921891Z res = mod(**inputs) 2025-12-04T09:44:44.1922415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mt5/modeling_mt5.py", line 1823, in forward 2025-12-04T09:44:44.1922662Z loss = loss_fct(lm_logits.view(-1, lm_logits.size(-1)), labels.view(-1)) 2025-12-04T09:44:44.1922679Z 2025-12-04T09:44:51.3594223Z Compilation time (from dynamo_timed): 30.012105776 2025-12-04T09:44:51.3882391Z pass 2025-12-04T09:44:51.3883479Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:44:51.3885387Z TIMING: _recursive_pre_grad_passes:0.37564 _recursive_joint_graph_passes:1.27343 _recursive_post_grad_passes:0.13157 async_compile.wait:1.07776 code_gen:6.95071 inductor_compile:11.90993 backend_compile:24.00914 gc:0.00083 entire_frame_compile:30.01211 total_wall_time:30.01211 2025-12-04T09:44:51.3887371Z STATS: call_* op count: 1189 | FakeTensorMode.__torch_dispatch__:32815 | FakeTensor.__torch_dispatch__:5771 | ProxyTorchDispatchMode.__torch_dispatch__:6849 2025-12-04T09:44:51.3888403Z Dynamo produced 1 graphs covering 1189 ops with 0 graph breaks (0 unique) 2025-12-04T09:44:54.4675330Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:44:54.4677036Z import pynvml # type: ignore[import] 2025-12-04T09:44:59.4163064Z 2025-12-04T09:44:59.4364416Z loading model: 0it [00:00, ?it/s]If you want to use `MegatronBertForCausalLM` as a standalone, add `is_decoder=True.` 2025-12-04T09:44:59.4366734Z WARNING:transformers.models.megatron_bert.modeling_megatron_bert:If you want to use `MegatronBertForCausalLM` as a standalone, add `is_decoder=True.` 2025-12-04T09:45:05.0517390Z 2025-12-04T09:45:05.0518021Z loading model: 0it [00:05, ?it/s] 2025-12-04T09:45:05.0558435Z cpu eval MegatronBertForCausalLM 2025-12-04T09:45:07.5478426Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:08.6229514Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:09.6812052Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:37.9195340Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9195970Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9196522Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9197106Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9197605Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9198069Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9198548Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9199013Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9199588Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9200034Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9200535Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9202144Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9202976Z res = mod(**inputs) 2025-12-04T09:45:37.9203923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9205046Z outputs = self.bert( 2025-12-04T09:45:37.9205905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9206830Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9207718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9208645Z layer_outputs = layer_module( 2025-12-04T09:45:37.9209372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9210148Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9210947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9211752Z return func(*args, **kwargs) 2025-12-04T09:45:37.9212635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9213973Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9214797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9215628Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9216651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9217659Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9218805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9219923Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9220828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9221687Z return self.act(input) 2025-12-04T09:45:37.9221990Z 2025-12-04T09:45:37.9222164Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9222720Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9223189Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9223645Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9224112Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9224563Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9225085Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9225536Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9225985Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9226441Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9226958Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9227693Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9228526Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9232940Z res = mod(**inputs) 2025-12-04T09:45:37.9233809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9234738Z outputs = self.bert( 2025-12-04T09:45:37.9235613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9236531Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9237673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9238661Z layer_outputs = layer_module( 2025-12-04T09:45:37.9239520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9240418Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9241530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9242460Z return func(*args, **kwargs) 2025-12-04T09:45:37.9243530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9244665Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9245505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9246333Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9247266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9248286Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9249233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9250331Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9251094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9251852Z return self.act(input) 2025-12-04T09:45:37.9252063Z 2025-12-04T09:45:37.9252226Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9252609Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9253003Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9253533Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9254051Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9254640Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9255125Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9255617Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9256081Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9256548Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9256933Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9257507Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9258462Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9259365Z res = mod(**inputs) 2025-12-04T09:45:37.9260483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9261526Z outputs = self.bert( 2025-12-04T09:45:37.9262709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9264000Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9265061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9266262Z layer_outputs = layer_module( 2025-12-04T09:45:37.9267416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9268342Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9269348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9270397Z return func(*args, **kwargs) 2025-12-04T09:45:37.9271934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9273092Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9274186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9275240Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9276365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9277802Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9279393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9280581Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9281688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9282518Z return self.act(input) 2025-12-04T09:45:37.9282859Z 2025-12-04T09:45:37.9283004Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9283396Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9283805Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9284461Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9284992Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9297749Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9298287Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9298698Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9299242Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9299791Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9300420Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9301366Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9302317Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9303344Z res = mod(**inputs) 2025-12-04T09:45:37.9304454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9305654Z outputs = self.bert( 2025-12-04T09:45:37.9306720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9307932Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9308932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9309836Z layer_outputs = layer_module( 2025-12-04T09:45:37.9310543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9311301Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9312099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9312908Z return func(*args, **kwargs) 2025-12-04T09:45:37.9313794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9314722Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9315583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9316436Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9317409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9318453Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9319524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9320492Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9321277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9321974Z return self.act(input) 2025-12-04T09:45:37.9322198Z 2025-12-04T09:45:37.9322339Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9322739Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9323116Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9323504Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9323894Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9324267Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9324648Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9325034Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9325418Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9325966Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9326359Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9326806Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9327522Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9328183Z res = mod(**inputs) 2025-12-04T09:45:37.9329099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9329999Z outputs = self.bert( 2025-12-04T09:45:37.9330819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9331762Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9332637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9333587Z layer_outputs = layer_module( 2025-12-04T09:45:37.9334299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9335052Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9335837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9336613Z return func(*args, **kwargs) 2025-12-04T09:45:37.9337474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9338390Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9339240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9340055Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9341005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9342022Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9342957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9343926Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9344707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9345414Z return self.act(input) 2025-12-04T09:45:37.9345618Z 2025-12-04T09:45:37.9345758Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9346156Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9346546Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9347043Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9347751Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9348154Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9348553Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9348937Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9349334Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9349734Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9350121Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9350575Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9351330Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9351994Z res = mod(**inputs) 2025-12-04T09:45:37.9352846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9353908Z outputs = self.bert( 2025-12-04T09:45:37.9354850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9355764Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9356671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9357602Z layer_outputs = layer_module( 2025-12-04T09:45:37.9358384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9359225Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9360043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9360895Z return func(*args, **kwargs) 2025-12-04T09:45:37.9361765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9362711Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9363649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9364499Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9365455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9366633Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9367579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9368546Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9369318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9370028Z return self.act(input) 2025-12-04T09:45:37.9370236Z 2025-12-04T09:45:37.9370398Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9370778Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9371172Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9371562Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9371934Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9372324Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9372715Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9373103Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9373472Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9373857Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9374243Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9374664Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9375397Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9376052Z res = mod(**inputs) 2025-12-04T09:45:37.9376866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9377752Z outputs = self.bert( 2025-12-04T09:45:37.9378584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9379480Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9380340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9381233Z layer_outputs = layer_module( 2025-12-04T09:45:37.9381942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9382691Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9383464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9384252Z return func(*args, **kwargs) 2025-12-04T09:45:37.9385109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9386008Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9386953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9388033Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9389009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9390075Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9391052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9392140Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9392949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9393664Z return self.act(input) 2025-12-04T09:45:37.9393893Z 2025-12-04T09:45:37.9394040Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9394455Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9394842Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9395245Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9395648Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9396031Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9396431Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9396829Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9397231Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9397614Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9398013Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9398583Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9399297Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9399958Z res = mod(**inputs) 2025-12-04T09:45:37.9401335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9402268Z outputs = self.bert( 2025-12-04T09:45:37.9403128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9404058Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9404967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9405873Z layer_outputs = layer_module( 2025-12-04T09:45:37.9406610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9407382Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9408193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9408981Z return func(*args, **kwargs) 2025-12-04T09:45:37.9409861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9410801Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9411643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9412493Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9413564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9414581Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9415513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9421421Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9422352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9423089Z return self.act(input) 2025-12-04T09:45:37.9423299Z 2025-12-04T09:45:37.9423447Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9423854Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9424305Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9424685Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9425080Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9425477Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9425946Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9426343Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9426863Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9427267Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9427649Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9428116Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9428870Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9429537Z res = mod(**inputs) 2025-12-04T09:45:37.9430391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9431310Z outputs = self.bert( 2025-12-04T09:45:37.9432148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9433081Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9433985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9434916Z layer_outputs = layer_module( 2025-12-04T09:45:37.9435629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9436404Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9437208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9438012Z return func(*args, **kwargs) 2025-12-04T09:45:37.9438975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9439897Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9440743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9441558Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9442501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9443524Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9444477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9445432Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9446212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9446927Z return self.act(input) 2025-12-04T09:45:37.9447133Z 2025-12-04T09:45:37.9447291Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9447670Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9448060Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9448444Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9448812Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9449190Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9449565Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9449977Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9450353Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9450735Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9451121Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9451544Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9452265Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9452958Z res = mod(**inputs) 2025-12-04T09:45:37.9453836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9454737Z outputs = self.bert( 2025-12-04T09:45:37.9455564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9456459Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9457307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9458204Z layer_outputs = layer_module( 2025-12-04T09:45:37.9458904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9459635Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9460424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9461204Z return func(*args, **kwargs) 2025-12-04T09:45:37.9462062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9462955Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9463785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9464611Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9465547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9466547Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9467774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9468775Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9469569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9470305Z return self.act(input) 2025-12-04T09:45:37.9470528Z 2025-12-04T09:45:37.9470670Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9471074Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9471454Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9471850Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9472226Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9472596Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9472974Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9473350Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9473719Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9474096Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9474472Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9474903Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9475626Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9476286Z res = mod(**inputs) 2025-12-04T09:45:37.9477118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9478007Z outputs = self.bert( 2025-12-04T09:45:37.9483916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9484833Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9485711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9486649Z layer_outputs = layer_module( 2025-12-04T09:45:37.9487364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9488181Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9488966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9489751Z return func(*args, **kwargs) 2025-12-04T09:45:37.9490627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9491670Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9492477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9493288Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9494213Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9495205Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9496122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9497076Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9497843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9498529Z return self.act(input) 2025-12-04T09:45:37.9498735Z 2025-12-04T09:45:37.9498866Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9499235Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9499600Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9499958Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9500331Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9500697Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9501609Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9501990Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9502382Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9502751Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9503131Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9503571Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9504300Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9504955Z res = mod(**inputs) 2025-12-04T09:45:37.9505789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9506695Z outputs = self.bert( 2025-12-04T09:45:37.9507628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9508550Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9509433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9510332Z layer_outputs = layer_module( 2025-12-04T09:45:37.9511027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9511789Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9512671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9513449Z return func(*args, **kwargs) 2025-12-04T09:45:37.9514311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9515272Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9516120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9517045Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9517988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9519112Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9520034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9520973Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9521726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9522421Z return self.act(input) 2025-12-04T09:45:37.9522620Z 2025-12-04T09:45:37.9522766Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9523131Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9523504Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9523874Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9524232Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9524599Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9524965Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9525319Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9525685Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9526048Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9526404Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9526823Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9527524Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9528164Z res = mod(**inputs) 2025-12-04T09:45:37.9528963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9529831Z outputs = self.bert( 2025-12-04T09:45:37.9530647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9531508Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9532361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9533239Z layer_outputs = layer_module( 2025-12-04T09:45:37.9533925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9534643Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9535404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9536164Z return func(*args, **kwargs) 2025-12-04T09:45:37.9536995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9537890Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9538707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9539517Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9540427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9541675Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9542867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9543892Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9544666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9545461Z return self.act(input) 2025-12-04T09:45:37.9545668Z 2025-12-04T09:45:37.9545813Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9546203Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9546575Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9547109Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9547492Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9547866Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9548244Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9548621Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9548983Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9549362Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9549747Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9550165Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9550894Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9551562Z res = mod(**inputs) 2025-12-04T09:45:37.9552383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9553270Z outputs = self.bert( 2025-12-04T09:45:37.9554099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9555024Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9555910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9556827Z layer_outputs = layer_module( 2025-12-04T09:45:37.9557555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9558322Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9559207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9559982Z return func(*args, **kwargs) 2025-12-04T09:45:37.9560833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9561742Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9562563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9563390Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9564333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9565340Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9566287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9567257Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9568032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9568721Z return self.act(input) 2025-12-04T09:45:37.9568939Z 2025-12-04T09:45:37.9569132Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9569528Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9569906Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9570296Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9570683Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9571103Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9571469Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9571851Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9572235Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9572608Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9573077Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9573517Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9574225Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9574883Z res = mod(**inputs) 2025-12-04T09:45:37.9575712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9576601Z outputs = self.bert( 2025-12-04T09:45:37.9577412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9578317Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9579188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9580082Z layer_outputs = layer_module( 2025-12-04T09:45:37.9580765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9581510Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9582291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9583051Z return func(*args, **kwargs) 2025-12-04T09:45:37.9583898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9584808Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9585640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9586450Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9587674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9588722Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9589688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9590667Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9591468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9592194Z return self.act(input) 2025-12-04T09:45:37.9592402Z 2025-12-04T09:45:37.9592545Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9592946Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9593344Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9593727Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9594119Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9594516Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9594905Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9595284Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9595678Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9596070Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9596447Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9596944Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9597688Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9598353Z res = mod(**inputs) 2025-12-04T09:45:37.9599310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9600232Z outputs = self.bert( 2025-12-04T09:45:37.9601547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9602646Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9603543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9604628Z layer_outputs = layer_module( 2025-12-04T09:45:37.9605358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9606119Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9606928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9607738Z return func(*args, **kwargs) 2025-12-04T09:45:37.9608605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9609610Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9610473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9611319Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9612270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9613316Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9614281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9615272Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9616054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9616900Z return self.act(input) 2025-12-04T09:45:37.9617100Z 2025-12-04T09:45:37.9617254Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9617633Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9618013Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9618398Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9618764Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9619143Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9619527Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9619908Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9620274Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9620661Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9621049Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9621474Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9622196Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9622852Z res = mod(**inputs) 2025-12-04T09:45:37.9623662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9624550Z outputs = self.bert( 2025-12-04T09:45:37.9625375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9626268Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9627463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9628384Z layer_outputs = layer_module( 2025-12-04T09:45:37.9629114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9629963Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9630758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9631634Z return func(*args, **kwargs) 2025-12-04T09:45:37.9632518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9633436Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9634293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9635152Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9636115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9637152Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9638125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9639131Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9640028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9640720Z return self.act(input) 2025-12-04T09:45:37.9640941Z 2025-12-04T09:45:37.9641082Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9641473Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9641848Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9642238Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9642620Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9642992Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9643379Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9643773Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9644164Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9644538Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9644919Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9645360Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9646061Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9646714Z res = mod(**inputs) 2025-12-04T09:45:37.9647535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9648428Z outputs = self.bert( 2025-12-04T09:45:37.9649237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9650127Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9650999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9651867Z layer_outputs = layer_module( 2025-12-04T09:45:37.9652572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9653318Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9654099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9654860Z return func(*args, **kwargs) 2025-12-04T09:45:37.9655744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9656655Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9657473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9658329Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9659271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9660349Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9661279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9662240Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9663014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9663721Z return self.act(input) 2025-12-04T09:45:37.9663920Z 2025-12-04T09:45:37.9664061Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9664450Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9664836Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9665206Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9665586Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9665966Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9671383Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9671832Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9672225Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9672615Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9672993Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9673442Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9674182Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9674836Z res = mod(**inputs) 2025-12-04T09:45:37.9675682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9676597Z outputs = self.bert( 2025-12-04T09:45:37.9677444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9678347Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9679251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9680272Z layer_outputs = layer_module( 2025-12-04T09:45:37.9680960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9681708Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9682490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9683264Z return func(*args, **kwargs) 2025-12-04T09:45:37.9684097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9685008Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9685845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9686667Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9687587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9688598Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9689594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9690543Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9691317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9692054Z return self.act(input) 2025-12-04T09:45:37.9692254Z 2025-12-04T09:45:37.9692407Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9692789Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9693233Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9693619Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9693987Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9694371Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9694748Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9695115Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9695500Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9695880Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9696262Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9696680Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9697409Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9698065Z res = mod(**inputs) 2025-12-04T09:45:37.9698876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9699773Z outputs = self.bert( 2025-12-04T09:45:37.9700595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9702113Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9703005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9703939Z layer_outputs = layer_module( 2025-12-04T09:45:37.9704666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9705419Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9706230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9707137Z return func(*args, **kwargs) 2025-12-04T09:45:37.9708024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9708952Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9709810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9710665Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9711635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9712670Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9713646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9714645Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9715434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9716163Z return self.act(input) 2025-12-04T09:45:37.9716390Z 2025-12-04T09:45:37.9716534Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9716938Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9717323Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9717805Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9718200Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9718579Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9719079Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9719461Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9719829Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9720253Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9720631Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9721069Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9721841Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9722500Z res = mod(**inputs) 2025-12-04T09:45:37.9723322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9724189Z outputs = self.bert( 2025-12-04T09:45:37.9725015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9725912Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9726783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9727657Z layer_outputs = layer_module( 2025-12-04T09:45:37.9728356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9734035Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9734834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9735637Z return func(*args, **kwargs) 2025-12-04T09:45:37.9736514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9737462Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9738302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9739148Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9740113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9741157Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9742113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9743102Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9743898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9744629Z return self.act(input) 2025-12-04T09:45:37.9744836Z 2025-12-04T09:45:37.9744978Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9745380Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9745775Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9746159Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9746559Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9747059Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9747439Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9747832Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9748230Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9748609Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9749009Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9749459Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9750200Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9750940Z res = mod(**inputs) 2025-12-04T09:45:37.9751787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9752703Z outputs = self.bert( 2025-12-04T09:45:37.9753532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9754503Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9755459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9756381Z layer_outputs = layer_module( 2025-12-04T09:45:37.9757085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9757854Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9758759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9759536Z return func(*args, **kwargs) 2025-12-04T09:45:37.9760372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9761281Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9762113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9762923Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9763866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9764882Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9765823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9766775Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9767546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9768244Z return self.act(input) 2025-12-04T09:45:37.9768448Z 2025-12-04T09:45:37.9768599Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9768974Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9769359Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9769740Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9770113Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9770496Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9770878Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9771248Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9771631Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9772015Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9772384Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9772817Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9773544Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9774200Z res = mod(**inputs) 2025-12-04T09:45:37.9775009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9775902Z outputs = self.bert( 2025-12-04T09:45:37.9776740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9777619Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9778490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9779418Z layer_outputs = layer_module( 2025-12-04T09:45:37.9780126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9780855Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9781637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9782452Z return func(*args, **kwargs) 2025-12-04T09:45:37.9783308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9784265Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9785104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9785924Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9786941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9788174Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9789142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9790133Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9790913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9791776Z return self.act(input) 2025-12-04T09:45:37.9792036Z 2025-12-04T09:45:37.9792193Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9792597Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9792978Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9793371Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9793763Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9794146Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9794540Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9794933Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9795313Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9795710Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9796102Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9796533Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9797335Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9798010Z res = mod(**inputs) 2025-12-04T09:45:37.9798864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1056, in forward 2025-12-04T09:45:37.9799772Z outputs = self.bert( 2025-12-04T09:45:37.9800617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 848, in forward 2025-12-04T09:45:37.9801982Z encoder_outputs = self.encoder( 2025-12-04T09:45:37.9802868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 532, in forward 2025-12-04T09:45:37.9803796Z layer_outputs = layer_module( 2025-12-04T09:45:37.9804527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:45:37.9805292Z return super().__call__(*args, **kwargs) 2025-12-04T09:45:37.9806088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:45:37.9806890Z return func(*args, **kwargs) 2025-12-04T09:45:37.9807768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 469, in forward 2025-12-04T09:45:37.9808708Z layer_output = apply_chunking_to_forward( 2025-12-04T09:45:37.9820034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:45:37.9820970Z return forward_fn(*input_tensors) 2025-12-04T09:45:37.9821921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 476, in feed_forward_chunk 2025-12-04T09:45:37.9823051Z intermediate_output = self.intermediate(ln_output) 2025-12-04T09:45:37.9824078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 392, in forward 2025-12-04T09:45:37.9825025Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:45:37.9825794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:45:37.9826499Z return self.act(input) 2025-12-04T09:45:37.9826698Z 2025-12-04T09:45:37.9826965Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9827341Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9827900Z cudagraph partition due to non gpu ops 2025-12-04T09:45:37.9828345Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:45:37.9829062Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:45:37.9829741Z res = mod(**inputs) 2025-12-04T09:45:37.9830585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/megatron_bert/modeling_megatron_bert.py", line 1078, in forward 2025-12-04T09:45:37.9831493Z lm_loss = self.loss_function( 2025-12-04T09:45:37.9832249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:45:37.9833262Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:45:37.9834306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:45:37.9835392Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:45:37.9835926Z 2025-12-04T09:45:48.1124795Z Compilation time (from dynamo_timed): 36.729559427 2025-12-04T09:45:48.1204197Z pass 2025-12-04T09:45:48.1204821Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:48.1206542Z TIMING: _recursive_pre_grad_passes:0.09096 _recursive_joint_graph_passes:1.24184 _recursive_post_grad_passes:0.14954 async_compile.wait:1.17366 code_gen:8.66934 inductor_compile:14.78093 backend_compile:28.49163 gc:0.00071 entire_frame_compile:36.72956 total_wall_time:36.72956 2025-12-04T09:45:48.1208526Z STATS: call_* op count: 723 | FakeTensorMode.__torch_dispatch__:39096 | FakeTensor.__torch_dispatch__:6663 | ProxyTorchDispatchMode.__torch_dispatch__:6492 2025-12-04T09:45:48.1209582Z Dynamo produced 1 graphs covering 723 ops with 0 graph breaks (0 unique) 2025-12-04T09:45:51.3846025Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:45:51.3847682Z import pynvml # type: ignore[import] 2025-12-04T09:45:56.4295737Z 2025-12-04T09:45:57.3784389Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:45:57.3785083Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:45:57.3896135Z cpu eval MobileBertForMaskedLM 2025-12-04T09:45:57.8004751Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:58.0481966Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:45:58.4807335Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:46:49.8467659Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8468772Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8469593Z res = mod(**inputs) 2025-12-04T09:46:49.8470625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8472131Z outputs = self.mobilebert( 2025-12-04T09:46:49.8473362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 791, in forward 2025-12-04T09:46:49.8474512Z embedding_output = self.embeddings( 2025-12-04T09:46:49.8475611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 199, in forward 2025-12-04T09:46:49.8476685Z inputs_embeds = torch.cat( 2025-12-04T09:46:49.8476974Z 2025-12-04T09:46:49.8477198Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8477802Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8478686Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8479457Z res = mod(**inputs) 2025-12-04T09:46:49.8480559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8481663Z outputs = self.mobilebert( 2025-12-04T09:46:49.8482726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 791, in forward 2025-12-04T09:46:49.8483918Z embedding_output = self.embeddings( 2025-12-04T09:46:49.8484921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 215, in forward 2025-12-04T09:46:49.8485921Z embeddings = self.LayerNorm(embeddings) 2025-12-04T09:46:49.8486996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8488059Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8488418Z 2025-12-04T09:46:49.8488579Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8489434Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8490174Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8490853Z res = mod(**inputs) 2025-12-04T09:46:49.8491714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8492700Z outputs = self.mobilebert( 2025-12-04T09:46:49.8493563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8494583Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8495729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8496803Z layer_outputs = layer_module( 2025-12-04T09:46:49.8497864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.8499332Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.8500699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.8502376Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.8503639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.8504907Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.8506217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8507515Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8508051Z 2025-12-04T09:46:49.8508230Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8508851Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8509321Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8509792Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8510286Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8510879Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8511352Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8511893Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8512382Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8513062Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8513785Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8514761Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8515574Z res = mod(**inputs) 2025-12-04T09:46:49.8516557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8517779Z outputs = self.mobilebert( 2025-12-04T09:46:49.8518896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8520037Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8521074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8522235Z layer_outputs = layer_module( 2025-12-04T09:46:49.8523343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.8524666Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.8525860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.8527076Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.8528350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.8529512Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8530702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8531774Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8532156Z 2025-12-04T09:46:49.8532319Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8532834Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8533746Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8534510Z res = mod(**inputs) 2025-12-04T09:46:49.8535535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8536645Z outputs = self.mobilebert( 2025-12-04T09:46:49.8537683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8538757Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8539789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8540878Z layer_outputs = layer_module( 2025-12-04T09:46:49.8541903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8543075Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8544219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.8545408Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.8546645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8548134Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8548682Z 2025-12-04T09:46:49.8548837Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8549459Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8550329Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8551206Z res = mod(**inputs) 2025-12-04T09:46:49.8556310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8557226Z outputs = self.mobilebert( 2025-12-04T09:46:49.8558076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8559320Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8560468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8561580Z layer_outputs = layer_module( 2025-12-04T09:46:49.8562778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8563917Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8565053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.8566299Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.8567640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.8568896Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8570340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8571676Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8572058Z 2025-12-04T09:46:49.8572212Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8572736Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8573596Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8574396Z res = mod(**inputs) 2025-12-04T09:46:49.8575407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8576459Z outputs = self.mobilebert( 2025-12-04T09:46:49.8577481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8578839Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8580030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8581248Z layer_outputs = layer_module( 2025-12-04T09:46:49.8582418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8583832Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8584973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.8586227Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.8587770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8589150Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8589627Z 2025-12-04T09:46:49.8589797Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8590379Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8591390Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8592168Z res = mod(**inputs) 2025-12-04T09:46:49.8593194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8594319Z outputs = self.mobilebert( 2025-12-04T09:46:49.8595386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8596519Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8597521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8598634Z layer_outputs = layer_module( 2025-12-04T09:46:49.8599761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8601421Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8602566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.8603770Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.8605010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.8606189Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8607255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8608432Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8608838Z 2025-12-04T09:46:49.8609019Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8609580Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8610505Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8611329Z res = mod(**inputs) 2025-12-04T09:46:49.8612301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8613362Z outputs = self.mobilebert( 2025-12-04T09:46:49.8619203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8620111Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8621046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8622071Z layer_outputs = layer_module( 2025-12-04T09:46:49.8623021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8624199Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8625286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.8626506Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.8627906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8629069Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8629468Z 2025-12-04T09:46:49.8629632Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8630201Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8631100Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8631918Z res = mod(**inputs) 2025-12-04T09:46:49.8632929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8633935Z outputs = self.mobilebert( 2025-12-04T09:46:49.8634921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8635910Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8636952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8638074Z layer_outputs = layer_module( 2025-12-04T09:46:49.8639163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8640187Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8641236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.8642400Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.8643555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.8644683Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8645808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8646864Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8647203Z 2025-12-04T09:46:49.8647377Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8647880Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8648692Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8649485Z res = mod(**inputs) 2025-12-04T09:46:49.8650414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8651448Z outputs = self.mobilebert( 2025-12-04T09:46:49.8652504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8653537Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8654513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8655592Z layer_outputs = layer_module( 2025-12-04T09:46:49.8656583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.8657707Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.8658857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8659949Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8660327Z 2025-12-04T09:46:49.8660527Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8660999Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8661815Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8662646Z res = mod(**inputs) 2025-12-04T09:46:49.8663576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8664520Z outputs = self.mobilebert( 2025-12-04T09:46:49.8665577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8666581Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8667926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8668982Z layer_outputs = layer_module( 2025-12-04T09:46:49.8670010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.8671234Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.8672502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.8673644Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.8674796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8675871Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8676232Z 2025-12-04T09:46:49.8676401Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8677615Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8678358Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8679031Z res = mod(**inputs) 2025-12-04T09:46:49.8679841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8680742Z outputs = self.mobilebert( 2025-12-04T09:46:49.8681857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8682747Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8683631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8684830Z layer_outputs = layer_module( 2025-12-04T09:46:49.8685892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.8687067Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.8688305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.8689498Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.8690676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.8691754Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8692919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8693958Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8694342Z 2025-12-04T09:46:49.8694502Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8694999Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8695817Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8696584Z res = mod(**inputs) 2025-12-04T09:46:49.8697564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8698582Z outputs = self.mobilebert( 2025-12-04T09:46:49.8699583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8700601Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8702173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8703217Z layer_outputs = layer_module( 2025-12-04T09:46:49.8704372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.8705746Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.8707135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.8708249Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.8709431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.8710453Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.8711519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8712633Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8712954Z 2025-12-04T09:46:49.8713182Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8713625Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8714097Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8714573Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8715002Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8715453Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8715904Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8716398Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8716833Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8717325Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8717828Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8718798Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8719631Z res = mod(**inputs) 2025-12-04T09:46:49.8720572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8721589Z outputs = self.mobilebert( 2025-12-04T09:46:49.8722519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8723580Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8724598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8725572Z layer_outputs = layer_module( 2025-12-04T09:46:49.8726557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.8727633Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.8728653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.8729746Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.8730822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.8732009Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8733275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8734294Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8734607Z 2025-12-04T09:46:49.8734816Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8735385Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8736225Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8737002Z res = mod(**inputs) 2025-12-04T09:46:49.8738070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8739435Z outputs = self.mobilebert( 2025-12-04T09:46:49.8740284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8741195Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8742078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8742977Z layer_outputs = layer_module( 2025-12-04T09:46:49.8743991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8744944Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8746009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.8747257Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.8748380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8749497Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8749925Z 2025-12-04T09:46:49.8750087Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8750575Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8751461Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8752270Z res = mod(**inputs) 2025-12-04T09:46:49.8753077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8754115Z outputs = self.mobilebert( 2025-12-04T09:46:49.8755117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8756235Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8757240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8758289Z layer_outputs = layer_module( 2025-12-04T09:46:49.8759357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8760409Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8761441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.8762565Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.8763736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.8764841Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8766028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8768417Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8768715Z 2025-12-04T09:46:49.8768959Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8769438Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8770251Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8771072Z res = mod(**inputs) 2025-12-04T09:46:49.8772001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8773011Z outputs = self.mobilebert( 2025-12-04T09:46:49.8774059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8775057Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8776034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8777120Z layer_outputs = layer_module( 2025-12-04T09:46:49.8778094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8779158Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8780242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.8781366Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.8782587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8783652Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8784082Z 2025-12-04T09:46:49.8784271Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8784804Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8785644Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8786391Z res = mod(**inputs) 2025-12-04T09:46:49.8787580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8788590Z outputs = self.mobilebert( 2025-12-04T09:46:49.8789571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8790613Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8791650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8792670Z layer_outputs = layer_module( 2025-12-04T09:46:49.8793689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8794747Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8795812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.8796999Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.8798143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.8799422Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8800482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8807291Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8807597Z 2025-12-04T09:46:49.8807743Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8808206Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8809063Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8809825Z res = mod(**inputs) 2025-12-04T09:46:49.8810647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8811609Z outputs = self.mobilebert( 2025-12-04T09:46:49.8812470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8813384Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8814330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8815237Z layer_outputs = layer_module( 2025-12-04T09:46:49.8816108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8817032Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8817964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.8819038Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.8819984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8820905Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8821243Z 2025-12-04T09:46:49.8821388Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8821829Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8822543Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8823199Z res = mod(**inputs) 2025-12-04T09:46:49.8823995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8824869Z outputs = self.mobilebert( 2025-12-04T09:46:49.8825682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8826566Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8827802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8828691Z layer_outputs = layer_module( 2025-12-04T09:46:49.8829544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8830480Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8831415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.8832412Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.8833416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.8834416Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8835426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8836351Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8836667Z 2025-12-04T09:46:49.8836812Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8837264Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8838006Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8838770Z res = mod(**inputs) 2025-12-04T09:46:49.8839611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8840479Z outputs = self.mobilebert( 2025-12-04T09:46:49.8841293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8842201Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8843048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8843919Z layer_outputs = layer_module( 2025-12-04T09:46:49.8844805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.8845765Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.8846722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8847664Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8847989Z 2025-12-04T09:46:49.8848128Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8848565Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8849283Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8849927Z res = mod(**inputs) 2025-12-04T09:46:49.8850724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8851594Z outputs = self.mobilebert( 2025-12-04T09:46:49.8852422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8853276Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8854122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8854988Z layer_outputs = layer_module( 2025-12-04T09:46:49.8855827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.8856857Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.8857913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.8858888Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.8859851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8860749Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8861047Z 2025-12-04T09:46:49.8861189Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8861630Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8862333Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8862980Z res = mod(**inputs) 2025-12-04T09:46:49.8863857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8869902Z outputs = self.mobilebert( 2025-12-04T09:46:49.8870751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8871653Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8872526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8873407Z layer_outputs = layer_module( 2025-12-04T09:46:49.8874351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.8875433Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.8876518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.8877542Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.8878606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.8879715Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8880700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8881599Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8881906Z 2025-12-04T09:46:49.8882052Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8882492Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8883197Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8883853Z res = mod(**inputs) 2025-12-04T09:46:49.8884656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8885526Z outputs = self.mobilebert( 2025-12-04T09:46:49.8886343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8887219Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8888072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8888947Z layer_outputs = layer_module( 2025-12-04T09:46:49.8889774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.8890826Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.8891898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.8892822Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.8893761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.8894660Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.8895546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8896437Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8896743Z 2025-12-04T09:46:49.8896883Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8897279Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8897669Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8898043Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8898427Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8898809Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8899173Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8899553Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8899933Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8900300Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8901105Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8902043Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8902718Z res = mod(**inputs) 2025-12-04T09:46:49.8903624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8904521Z outputs = self.mobilebert( 2025-12-04T09:46:49.8905374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8906302Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8907292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8908284Z layer_outputs = layer_module( 2025-12-04T09:46:49.8909151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.8910052Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.8910955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.8911949Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.8912932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.8913938Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8914943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8915869Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8916170Z 2025-12-04T09:46:49.8916315Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8916768Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8917507Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8918185Z res = mod(**inputs) 2025-12-04T09:46:49.8919086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8919955Z outputs = self.mobilebert( 2025-12-04T09:46:49.8920783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8921639Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8922482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8923354Z layer_outputs = layer_module( 2025-12-04T09:46:49.8924192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8925088Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8925993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.8927361Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.8928336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8929298Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8929651Z 2025-12-04T09:46:49.8929797Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8930252Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8931040Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8931718Z res = mod(**inputs) 2025-12-04T09:46:49.8932540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8933441Z outputs = self.mobilebert( 2025-12-04T09:46:49.8934338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8935239Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8936119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8937044Z layer_outputs = layer_module( 2025-12-04T09:46:49.8937889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8938982Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8939888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.8940853Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.8941827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.8942799Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8943768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8944656Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8944955Z 2025-12-04T09:46:49.8945096Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8945542Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8946271Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8947009Z res = mod(**inputs) 2025-12-04T09:46:49.8948001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8948895Z outputs = self.mobilebert( 2025-12-04T09:46:49.8949735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8950636Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8951515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8952407Z layer_outputs = layer_module( 2025-12-04T09:46:49.8953253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8954193Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8955125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.8956101Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.8957052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8958030Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8958366Z 2025-12-04T09:46:49.8958527Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8958965Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8959802Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8960455Z res = mod(**inputs) 2025-12-04T09:46:49.8961257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8962111Z outputs = self.mobilebert( 2025-12-04T09:46:49.8962945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8963821Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8964703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8965573Z layer_outputs = layer_module( 2025-12-04T09:46:49.8966408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8967347Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8968241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.8969295Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.8970269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.8971256Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.8972215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.8973123Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.8973410Z 2025-12-04T09:46:49.8973567Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8974013Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8974714Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8975361Z res = mod(**inputs) 2025-12-04T09:46:49.8976156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8977011Z outputs = self.mobilebert( 2025-12-04T09:46:49.8977839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8978711Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8979553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8980403Z layer_outputs = layer_module( 2025-12-04T09:46:49.8981236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8982136Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8983035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.8983983Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.8984920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.8985862Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.8986189Z 2025-12-04T09:46:49.8986330Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.8986880Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.8987829Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.8988503Z res = mod(**inputs) 2025-12-04T09:46:49.8989486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.8990389Z outputs = self.mobilebert( 2025-12-04T09:46:49.8991248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.8992133Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.8993007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.8993960Z layer_outputs = layer_module( 2025-12-04T09:46:49.8994886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.8995813Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.8996747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.8997788Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.8998856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.8999842Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9001300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9002250Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9002553Z 2025-12-04T09:46:49.9002715Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9003154Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9003895Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9004573Z res = mod(**inputs) 2025-12-04T09:46:49.9025601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9026488Z outputs = self.mobilebert( 2025-12-04T09:46:49.9027473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9028552Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9029416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9030328Z layer_outputs = layer_module( 2025-12-04T09:46:49.9031198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9032194Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9033170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9034153Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9034491Z 2025-12-04T09:46:49.9034657Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9035129Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9035862Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9036537Z res = mod(**inputs) 2025-12-04T09:46:49.9037362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9038250Z outputs = self.mobilebert( 2025-12-04T09:46:49.9039210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9040090Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9040945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9041796Z layer_outputs = layer_module( 2025-12-04T09:46:49.9042640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9043688Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9044750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9045823Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9046797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9047701Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9048037Z 2025-12-04T09:46:49.9048179Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9048623Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9049340Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9050069Z res = mod(**inputs) 2025-12-04T09:46:49.9050854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9056929Z outputs = self.mobilebert( 2025-12-04T09:46:49.9057794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9058695Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9059575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9060472Z layer_outputs = layer_module( 2025-12-04T09:46:49.9061345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9062412Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9063504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9064508Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9065516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9066513Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9067620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9068554Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9068854Z 2025-12-04T09:46:49.9069016Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9069461Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9070214Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9070887Z res = mod(**inputs) 2025-12-04T09:46:49.9071696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9072597Z outputs = self.mobilebert( 2025-12-04T09:46:49.9073457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9074355Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9075211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9076106Z layer_outputs = layer_module( 2025-12-04T09:46:49.9076965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9078053Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.9079230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.9080166Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.9081108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.9082047Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.9082936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9083877Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9084161Z 2025-12-04T09:46:49.9084318Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9084694Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9085087Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9085531Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9085905Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9086294Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9086677Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9087063Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9087432Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9087810Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9088247Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9088948Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9089601Z res = mod(**inputs) 2025-12-04T09:46:49.9090408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9091265Z outputs = self.mobilebert( 2025-12-04T09:46:49.9092098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9092971Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9093822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9094681Z layer_outputs = layer_module( 2025-12-04T09:46:49.9095517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.9096406Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.9097289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.9098242Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.9099209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.9100185Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9101819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9102767Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9103085Z 2025-12-04T09:46:49.9103229Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9103669Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9104387Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9105051Z res = mod(**inputs) 2025-12-04T09:46:49.9105861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9106846Z outputs = self.mobilebert( 2025-12-04T09:46:49.9107694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9108577Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9109437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9110399Z layer_outputs = layer_module( 2025-12-04T09:46:49.9111255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9112176Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9113104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9118944Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9120009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9120972Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9121309Z 2025-12-04T09:46:49.9121466Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9121908Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9122638Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9123304Z res = mod(**inputs) 2025-12-04T09:46:49.9124111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9124994Z outputs = self.mobilebert( 2025-12-04T09:46:49.9125841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9126824Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9127649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9128518Z layer_outputs = layer_module( 2025-12-04T09:46:49.9129332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9130221Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9131115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9132080Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9133050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9134008Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9134975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9135867Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9136151Z 2025-12-04T09:46:49.9136292Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9136712Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9137419Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9138047Z res = mod(**inputs) 2025-12-04T09:46:49.9138813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9139669Z outputs = self.mobilebert( 2025-12-04T09:46:49.9140490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9141361Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9142187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9143041Z layer_outputs = layer_module( 2025-12-04T09:46:49.9143862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9144784Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9145662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9146585Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9147827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9148790Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9149122Z 2025-12-04T09:46:49.9149347Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9149793Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9150526Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9151172Z res = mod(**inputs) 2025-12-04T09:46:49.9151978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9152874Z outputs = self.mobilebert( 2025-12-04T09:46:49.9153716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9154594Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9155462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9156334Z layer_outputs = layer_module( 2025-12-04T09:46:49.9157178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9158114Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9159140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9160111Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9161060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9162014Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9162974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9163861Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9164147Z 2025-12-04T09:46:49.9164286Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9164713Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9165417Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9166035Z res = mod(**inputs) 2025-12-04T09:46:49.9166820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9167677Z outputs = self.mobilebert( 2025-12-04T09:46:49.9168497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9169346Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9170184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9171038Z layer_outputs = layer_module( 2025-12-04T09:46:49.9171856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9172747Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9173627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9174598Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9175518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9176522Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9177202Z 2025-12-04T09:46:49.9177398Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9177842Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9178624Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9179292Z res = mod(**inputs) 2025-12-04T09:46:49.9180094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9181028Z outputs = self.mobilebert( 2025-12-04T09:46:49.9181877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9182758Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9183612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9184502Z layer_outputs = layer_module( 2025-12-04T09:46:49.9185350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9186259Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9187277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9188280Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9189272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9190250Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9191238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9192154Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9192447Z 2025-12-04T09:46:49.9192593Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9193023Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9193759Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9194417Z res = mod(**inputs) 2025-12-04T09:46:49.9195208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9196096Z outputs = self.mobilebert( 2025-12-04T09:46:49.9196947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9197833Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9198782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9199637Z layer_outputs = layer_module( 2025-12-04T09:46:49.9200464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9201978Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9202948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9203907Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9204236Z 2025-12-04T09:46:49.9204392Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9204912Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9205648Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9206312Z res = mod(**inputs) 2025-12-04T09:46:49.9207122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9208076Z outputs = self.mobilebert( 2025-12-04T09:46:49.9208919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9209883Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9210735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9211610Z layer_outputs = layer_module( 2025-12-04T09:46:49.9212453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9213520Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9214674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9215635Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9216585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9217477Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9217757Z 2025-12-04T09:46:49.9217890Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9218312Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9219011Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9219641Z res = mod(**inputs) 2025-12-04T09:46:49.9220422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9221276Z outputs = self.mobilebert( 2025-12-04T09:46:49.9222086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9222935Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9223766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9224615Z layer_outputs = layer_module( 2025-12-04T09:46:49.9225434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9226460Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9227859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9228851Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9229830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9230826Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9231822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9232743Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9233034Z 2025-12-04T09:46:49.9233175Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9233615Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9234338Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9235038Z res = mod(**inputs) 2025-12-04T09:46:49.9235835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9236714Z outputs = self.mobilebert( 2025-12-04T09:46:49.9237587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9238460Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9239569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9240459Z layer_outputs = layer_module( 2025-12-04T09:46:49.9241310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9242373Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.9243525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.9244477Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.9245433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.9246329Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.9247220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9248128Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9248418Z 2025-12-04T09:46:49.9248564Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9248945Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9249331Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9249712Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9250082Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9250461Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9250955Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9251309Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9251675Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9252031Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9252440Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9253148Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9253797Z res = mod(**inputs) 2025-12-04T09:46:49.9254578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9255423Z outputs = self.mobilebert( 2025-12-04T09:46:49.9256236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9257087Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9257916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9258758Z layer_outputs = layer_module( 2025-12-04T09:46:49.9259577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.9260453Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.9261310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.9262254Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.9263208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.9264215Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9265169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9266119Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9266401Z 2025-12-04T09:46:49.9266553Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9267087Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9268042Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9268710Z res = mod(**inputs) 2025-12-04T09:46:49.9269518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9270387Z outputs = self.mobilebert( 2025-12-04T09:46:49.9271231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9272117Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9272967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9273837Z layer_outputs = layer_module( 2025-12-04T09:46:49.9274680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9275609Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9276513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9277470Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9278421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9279478Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9279797Z 2025-12-04T09:46:49.9279931Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9280361Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9281070Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9281712Z res = mod(**inputs) 2025-12-04T09:46:49.9282485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9283347Z outputs = self.mobilebert( 2025-12-04T09:46:49.9284154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9284994Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9285827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9286677Z layer_outputs = layer_module( 2025-12-04T09:46:49.9287493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9288378Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9289262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9290226Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9291184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9292134Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9293093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9294010Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9294289Z 2025-12-04T09:46:49.9294433Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9294842Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9295571Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9296201Z res = mod(**inputs) 2025-12-04T09:46:49.9297021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9297876Z outputs = self.mobilebert( 2025-12-04T09:46:49.9298683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9299533Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9300358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9307007Z layer_outputs = layer_module( 2025-12-04T09:46:49.9307869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9308789Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9309701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9310670Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9311619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9312557Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9312895Z 2025-12-04T09:46:49.9313039Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9313472Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9314189Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9314839Z res = mod(**inputs) 2025-12-04T09:46:49.9315645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9316538Z outputs = self.mobilebert( 2025-12-04T09:46:49.9317384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9318278Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9319238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9320100Z layer_outputs = layer_module( 2025-12-04T09:46:49.9320925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9321834Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9322736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9323712Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9324663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9325638Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9326609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9327497Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9327896Z 2025-12-04T09:46:49.9328036Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9328476Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9329196Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9329836Z res = mod(**inputs) 2025-12-04T09:46:49.9330676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9331538Z outputs = self.mobilebert( 2025-12-04T09:46:49.9332439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9333297Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9334144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9335012Z layer_outputs = layer_module( 2025-12-04T09:46:49.9335836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9336744Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9337650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9338602Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9339531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9340472Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9340798Z 2025-12-04T09:46:49.9340952Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9341390Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9342088Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9342743Z res = mod(**inputs) 2025-12-04T09:46:49.9343537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9344391Z outputs = self.mobilebert( 2025-12-04T09:46:49.9345216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9346078Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9347019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9348069Z layer_outputs = layer_module( 2025-12-04T09:46:49.9348927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9349863Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9350783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9351776Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9352771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9353763Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9354744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9355671Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9355969Z 2025-12-04T09:46:49.9356113Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9356553Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9357266Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9357974Z res = mod(**inputs) 2025-12-04T09:46:49.9358791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9359752Z outputs = self.mobilebert( 2025-12-04T09:46:49.9360594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9361457Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9362352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9363195Z layer_outputs = layer_module( 2025-12-04T09:46:49.9369149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9370129Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9371101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9372044Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9372383Z 2025-12-04T09:46:49.9372526Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9372972Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9373681Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9374348Z res = mod(**inputs) 2025-12-04T09:46:49.9375156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9376042Z outputs = self.mobilebert( 2025-12-04T09:46:49.9376865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9377749Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9378612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9379500Z layer_outputs = layer_module( 2025-12-04T09:46:49.9380343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9381508Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9382563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9383510Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9384471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9385367Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9385649Z 2025-12-04T09:46:49.9385796Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9386212Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9386996Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9387841Z res = mod(**inputs) 2025-12-04T09:46:49.9388659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9389541Z outputs = self.mobilebert( 2025-12-04T09:46:49.9390390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9391290Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9392142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9393093Z layer_outputs = layer_module( 2025-12-04T09:46:49.9393941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9395009Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9396112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9397103Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9398153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9399247Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9400194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9401667Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9401985Z 2025-12-04T09:46:49.9402140Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9402580Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9403308Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9403968Z res = mod(**inputs) 2025-12-04T09:46:49.9404775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9405645Z outputs = self.mobilebert( 2025-12-04T09:46:49.9406491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9407369Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9408237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9409117Z layer_outputs = layer_module( 2025-12-04T09:46:49.9409966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9411046Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.9412112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.9413080Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.9414142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.9415027Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.9415879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9416766Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9417057Z 2025-12-04T09:46:49.9417192Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9417568Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9417938Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9418303Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9418670Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9419029Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9419402Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9419778Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9420138Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9420505Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9420927Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9421330Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9421538Z res = mod(**inputs) 2025-12-04T09:46:49.9422149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9422271Z outputs = self.mobilebert( 2025-12-04T09:46:49.9422929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9423057Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9423737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9423869Z layer_outputs = layer_module( 2025-12-04T09:46:49.9424477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.9424634Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.9425239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.9425454Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.9426117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.9426359Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9427288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9427622Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9427639Z 2025-12-04T09:46:49.9427775Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9427982Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9428406Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9428520Z res = mod(**inputs) 2025-12-04T09:46:49.9429151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9429273Z outputs = self.mobilebert( 2025-12-04T09:46:49.9429908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9430037Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9430665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9430797Z layer_outputs = layer_module( 2025-12-04T09:46:49.9431487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9431663Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9432288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9432494Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9433132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9433337Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9433354Z 2025-12-04T09:46:49.9433507Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9433705Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9434120Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9434243Z res = mod(**inputs) 2025-12-04T09:46:49.9434874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9435047Z outputs = self.mobilebert( 2025-12-04T09:46:49.9435688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9435821Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9436495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9436619Z layer_outputs = layer_module( 2025-12-04T09:46:49.9437307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9437492Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9438117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9438369Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9439109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9439339Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9439959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9440119Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9440134Z 2025-12-04T09:46:49.9440274Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9440466Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9440869Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9440994Z res = mod(**inputs) 2025-12-04T09:46:49.9441606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9441733Z outputs = self.mobilebert( 2025-12-04T09:46:49.9442352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9442479Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9443093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9443209Z layer_outputs = layer_module( 2025-12-04T09:46:49.9443821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9443990Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9444597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9444797Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9445408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9445602Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9445619Z 2025-12-04T09:46:49.9445760Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9445944Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9446348Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9446466Z res = mod(**inputs) 2025-12-04T09:46:49.9447074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9447205Z outputs = self.mobilebert( 2025-12-04T09:46:49.9447811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9447968Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9448583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9448704Z layer_outputs = layer_module( 2025-12-04T09:46:49.9449338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9449513Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9450181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9450421Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9451024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9451253Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9451871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9452029Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9452046Z 2025-12-04T09:46:49.9452198Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9452380Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9452786Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9452905Z res = mod(**inputs) 2025-12-04T09:46:49.9453515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9453635Z outputs = self.mobilebert( 2025-12-04T09:46:49.9454249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9454378Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9454998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9455119Z layer_outputs = layer_module( 2025-12-04T09:46:49.9455725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9455901Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9456508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9456718Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9457323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9457519Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9457534Z 2025-12-04T09:46:49.9457681Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9457863Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9458260Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9458386Z res = mod(**inputs) 2025-12-04T09:46:49.9458992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9459129Z outputs = self.mobilebert( 2025-12-04T09:46:49.9459737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9459864Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9460479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9460631Z layer_outputs = layer_module( 2025-12-04T09:46:49.9461249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9461444Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9462053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9462294Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9462957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9463183Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9463803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9463961Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9463975Z 2025-12-04T09:46:49.9464124Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9464308Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9464713Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9464832Z res = mod(**inputs) 2025-12-04T09:46:49.9465442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9465571Z outputs = self.mobilebert( 2025-12-04T09:46:49.9466174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9466298Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9467033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9467329Z layer_outputs = layer_module( 2025-12-04T09:46:49.9467960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9468199Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9468822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9469037Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9469053Z 2025-12-04T09:46:49.9469194Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9469382Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9469810Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9469925Z res = mod(**inputs) 2025-12-04T09:46:49.9470564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9470688Z outputs = self.mobilebert( 2025-12-04T09:46:49.9471311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9471455Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9472077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9472204Z layer_outputs = layer_module( 2025-12-04T09:46:49.9472839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9473153Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9473835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9474064Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9474690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9474897Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9474913Z 2025-12-04T09:46:49.9475049Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9475250Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9475713Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9475825Z res = mod(**inputs) 2025-12-04T09:46:49.9476466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9476592Z outputs = self.mobilebert( 2025-12-04T09:46:49.9477217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9477357Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9477986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9478126Z layer_outputs = layer_module( 2025-12-04T09:46:49.9478758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9479188Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9479813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9480037Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9480659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9480888Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9481489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9481660Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9481675Z 2025-12-04T09:46:49.9481811Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9482011Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9482418Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9482522Z res = mod(**inputs) 2025-12-04T09:46:49.9483139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9483264Z outputs = self.mobilebert( 2025-12-04T09:46:49.9483868Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9484002Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9484610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9484741Z layer_outputs = layer_module( 2025-12-04T09:46:49.9485349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9485658Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.9486277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.9486501Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.9487116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.9487264Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.9487902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9488070Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9488085Z 2025-12-04T09:46:49.9488221Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9488408Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9488603Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9488749Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9488888Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9489016Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9489527Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9489671Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9489806Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9490030Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9490237Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9490658Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9490767Z res = mod(**inputs) 2025-12-04T09:46:49.9491416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9491544Z outputs = self.mobilebert( 2025-12-04T09:46:49.9492186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9492318Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9492948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9493089Z layer_outputs = layer_module( 2025-12-04T09:46:49.9493780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.9493946Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.9494577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.9494809Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.9495452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.9495688Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9496318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9496485Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9496500Z 2025-12-04T09:46:49.9496633Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9496837Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9497261Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9497371Z res = mod(**inputs) 2025-12-04T09:46:49.9498006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9498137Z outputs = self.mobilebert( 2025-12-04T09:46:49.9498777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9498908Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9499581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9499713Z layer_outputs = layer_module( 2025-12-04T09:46:49.9500336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9500536Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9501594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9501932Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9502567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9502767Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9502783Z 2025-12-04T09:46:49.9502931Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9503131Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9503542Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9503671Z res = mod(**inputs) 2025-12-04T09:46:49.9504301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9504432Z outputs = self.mobilebert( 2025-12-04T09:46:49.9505084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9505214Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9505849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9505992Z layer_outputs = layer_module( 2025-12-04T09:46:49.9506619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9506907Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9507535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9507780Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9508413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9508653Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9509294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9509456Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9509471Z 2025-12-04T09:46:49.9509615Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9509820Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9510232Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9510358Z res = mod(**inputs) 2025-12-04T09:46:49.9510982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9511110Z outputs = self.mobilebert( 2025-12-04T09:46:49.9511752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9511886Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9512510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9512638Z layer_outputs = layer_module( 2025-12-04T09:46:49.9513310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9513487Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9514106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9514347Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9514990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9515246Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9515262Z 2025-12-04T09:46:49.9515411Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9515599Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9516011Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9516130Z res = mod(**inputs) 2025-12-04T09:46:49.9516760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9516885Z outputs = self.mobilebert( 2025-12-04T09:46:49.9517519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9517647Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9518296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9518529Z layer_outputs = layer_module( 2025-12-04T09:46:49.9519136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9519305Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9519907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9520148Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9520750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9520981Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9521605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9521762Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9521777Z 2025-12-04T09:46:49.9521908Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9522103Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9522503Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9522625Z res = mod(**inputs) 2025-12-04T09:46:49.9523232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9523351Z outputs = self.mobilebert( 2025-12-04T09:46:49.9523977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9524102Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9524725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9524847Z layer_outputs = layer_module( 2025-12-04T09:46:49.9525453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9525627Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9526263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9526462Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9527079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9527303Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9527318Z 2025-12-04T09:46:49.9527468Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9527699Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9528101Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9528222Z res = mod(**inputs) 2025-12-04T09:46:49.9528829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9528960Z outputs = self.mobilebert( 2025-12-04T09:46:49.9529568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9529689Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9530307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9530426Z layer_outputs = layer_module( 2025-12-04T09:46:49.9531037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9531207Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9531809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9532053Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9532668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9532896Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9533512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9533674Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9533688Z 2025-12-04T09:46:49.9533835Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9534024Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9534422Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9534548Z res = mod(**inputs) 2025-12-04T09:46:49.9535153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9535278Z outputs = self.mobilebert( 2025-12-04T09:46:49.9535900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9536026Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9536648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9536766Z layer_outputs = layer_module( 2025-12-04T09:46:49.9537376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9537602Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9538206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9538471Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9538486Z 2025-12-04T09:46:49.9538621Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9538804Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9539211Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9539350Z res = mod(**inputs) 2025-12-04T09:46:49.9539958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9540086Z outputs = self.mobilebert( 2025-12-04T09:46:49.9540751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9540891Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9541495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9541619Z layer_outputs = layer_module( 2025-12-04T09:46:49.9542228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9542532Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9543150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9543375Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9543973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9544143Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9544158Z 2025-12-04T09:46:49.9544292Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9544471Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9544883Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9544987Z res = mod(**inputs) 2025-12-04T09:46:49.9545598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9545723Z outputs = self.mobilebert( 2025-12-04T09:46:49.9546329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9546460Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9547329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9547474Z layer_outputs = layer_module( 2025-12-04T09:46:49.9548104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9548415Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9549057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9549290Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9549932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9550163Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9550787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9550977Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9550998Z 2025-12-04T09:46:49.9551189Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9551424Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9556753Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9556868Z res = mod(**inputs) 2025-12-04T09:46:49.9557508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9557688Z outputs = self.mobilebert( 2025-12-04T09:46:49.9558320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9558518Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9559144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9559282Z layer_outputs = layer_module( 2025-12-04T09:46:49.9559906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9560229Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.9560869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.9561068Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.9561694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.9561860Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.9562484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9562658Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9562673Z 2025-12-04T09:46:49.9562818Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9562953Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9563094Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9563229Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9563366Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9563516Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9563651Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9563800Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9563932Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9564066Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9564267Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9564683Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9564790Z res = mod(**inputs) 2025-12-04T09:46:49.9565432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9565564Z outputs = self.mobilebert( 2025-12-04T09:46:49.9566204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9566456Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9567064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9567203Z layer_outputs = layer_module( 2025-12-04T09:46:49.9567814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.9567959Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.9568581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.9568802Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.9569461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.9569688Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9570298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9570494Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9570510Z 2025-12-04T09:46:49.9570641Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9570893Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9571298Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9571404Z res = mod(**inputs) 2025-12-04T09:46:49.9572034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9572161Z outputs = self.mobilebert( 2025-12-04T09:46:49.9572772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9572910Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9573518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9573643Z layer_outputs = layer_module( 2025-12-04T09:46:49.9574252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9574413Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9575024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9575222Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9575833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9576026Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9576043Z 2025-12-04T09:46:49.9576174Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9576367Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9576770Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9576883Z res = mod(**inputs) 2025-12-04T09:46:49.9577499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9577615Z outputs = self.mobilebert( 2025-12-04T09:46:49.9578231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9578360Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9578964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9579098Z layer_outputs = layer_module( 2025-12-04T09:46:49.9579701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9579880Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9580488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9580719Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9581341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9581595Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9582210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9582364Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9582408Z 2025-12-04T09:46:49.9582544Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9582737Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9583136Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9583293Z res = mod(**inputs) 2025-12-04T09:46:49.9583914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9584035Z outputs = self.mobilebert( 2025-12-04T09:46:49.9584659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9584783Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9585390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9585522Z layer_outputs = layer_module( 2025-12-04T09:46:49.9586123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9586281Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9586989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9587364Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9587992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9588189Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9588205Z 2025-12-04T09:46:49.9588344Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9588543Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9588957Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9589078Z res = mod(**inputs) 2025-12-04T09:46:49.9589705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9589835Z outputs = self.mobilebert( 2025-12-04T09:46:49.9590480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9590606Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9591232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9591374Z layer_outputs = layer_module( 2025-12-04T09:46:49.9592004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9592184Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9592813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9593051Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9604617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9604906Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9605572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9605870Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9605888Z 2025-12-04T09:46:49.9606034Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9606242Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9606660Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9606822Z res = mod(**inputs) 2025-12-04T09:46:49.9607470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9607675Z outputs = self.mobilebert( 2025-12-04T09:46:49.9608321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9608454Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9609088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9609229Z layer_outputs = layer_module( 2025-12-04T09:46:49.9609850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9610034Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9610662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9610871Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9611514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9611715Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9611732Z 2025-12-04T09:46:49.9611876Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9612079Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9612497Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9612623Z res = mod(**inputs) 2025-12-04T09:46:49.9613254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9613388Z outputs = self.mobilebert( 2025-12-04T09:46:49.9619046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9619185Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9619827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9619950Z layer_outputs = layer_module( 2025-12-04T09:46:49.9620575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9620762Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9621387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9621630Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9622269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9622507Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9623143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9623305Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9623321Z 2025-12-04T09:46:49.9623459Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9623722Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9624137Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9624265Z res = mod(**inputs) 2025-12-04T09:46:49.9624899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9625057Z outputs = self.mobilebert( 2025-12-04T09:46:49.9625704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9625895Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9626522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9626659Z layer_outputs = layer_module( 2025-12-04T09:46:49.9627384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9627623Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9628248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9628451Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9628467Z 2025-12-04T09:46:49.9628622Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9628813Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9629247Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9629363Z res = mod(**inputs) 2025-12-04T09:46:49.9629986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9630122Z outputs = self.mobilebert( 2025-12-04T09:46:49.9630753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9630882Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9631516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9631642Z layer_outputs = layer_module( 2025-12-04T09:46:49.9632278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9632600Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9633226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9633470Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9634098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9634272Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9634288Z 2025-12-04T09:46:49.9634429Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9634622Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9635052Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9635166Z res = mod(**inputs) 2025-12-04T09:46:49.9635792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9635926Z outputs = self.mobilebert( 2025-12-04T09:46:49.9636552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9636733Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9637362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9637488Z layer_outputs = layer_module( 2025-12-04T09:46:49.9638123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9638579Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9639253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9639478Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9640092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9640334Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9640943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9641117Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9641132Z 2025-12-04T09:46:49.9641271Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9641457Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9641873Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9641977Z res = mod(**inputs) 2025-12-04T09:46:49.9642587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9642717Z outputs = self.mobilebert( 2025-12-04T09:46:49.9643323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9643460Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9644067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9644182Z layer_outputs = layer_module( 2025-12-04T09:46:49.9644802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9645110Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.9645731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.9645924Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.9646536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.9646696Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.9647303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9647458Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9647489Z 2025-12-04T09:46:49.9647624Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9647757Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9647904Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9648035Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9648170Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9648311Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9648441Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9648569Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9648713Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9648842Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9649058Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9649473Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9649583Z res = mod(**inputs) 2025-12-04T09:46:49.9650204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9650356Z outputs = self.mobilebert( 2025-12-04T09:46:49.9650965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9651159Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9651765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9651896Z layer_outputs = layer_module( 2025-12-04T09:46:49.9652505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.9652653Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.9653271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.9653495Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.9654097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.9654340Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9654948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9655113Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9655128Z 2025-12-04T09:46:49.9655265Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9655453Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9655864Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9655975Z res = mod(**inputs) 2025-12-04T09:46:49.9656595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9656720Z outputs = self.mobilebert( 2025-12-04T09:46:49.9657334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9657475Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9658078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9658200Z layer_outputs = layer_module( 2025-12-04T09:46:49.9658816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9658982Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9659604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9659805Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9660414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9660628Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9660643Z 2025-12-04T09:46:49.9660778Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9660975Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9661374Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9661514Z res = mod(**inputs) 2025-12-04T09:46:49.9662139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9662263Z outputs = self.mobilebert( 2025-12-04T09:46:49.9662871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9663878Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9664566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9664705Z layer_outputs = layer_module( 2025-12-04T09:46:49.9665310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9665474Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9666093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9666323Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9667079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9667490Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9668114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9668299Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9668315Z 2025-12-04T09:46:49.9668456Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9668662Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9669080Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9669196Z res = mod(**inputs) 2025-12-04T09:46:49.9669844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9669970Z outputs = self.mobilebert( 2025-12-04T09:46:49.9670596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9670742Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9671370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9671508Z layer_outputs = layer_module( 2025-12-04T09:46:49.9672133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9672301Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9672938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9673142Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9673773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9673974Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9673990Z 2025-12-04T09:46:49.9674129Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9674332Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9674746Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9674857Z res = mod(**inputs) 2025-12-04T09:46:49.9675487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9675651Z outputs = self.mobilebert( 2025-12-04T09:46:49.9676362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9676600Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9677226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9677398Z layer_outputs = layer_module( 2025-12-04T09:46:49.9678076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9678247Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9678884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9679121Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9679760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9679992Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9680615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9680793Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9680814Z 2025-12-04T09:46:49.9681013Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9681222Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9681639Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9681750Z res = mod(**inputs) 2025-12-04T09:46:49.9682394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9682525Z outputs = self.mobilebert( 2025-12-04T09:46:49.9683168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9683297Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9683921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9684063Z layer_outputs = layer_module( 2025-12-04T09:46:49.9684693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9684865Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9685499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9685708Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9686353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9686553Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9686568Z 2025-12-04T09:46:49.9686705Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9686906Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9687316Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9687443Z res = mod(**inputs) 2025-12-04T09:46:49.9688077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9688201Z outputs = self.mobilebert( 2025-12-04T09:46:49.9688931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9689093Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9689697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9689825Z layer_outputs = layer_module( 2025-12-04T09:46:49.9690422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9690625Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9691283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9691511Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9692122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9692348Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9692961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9693120Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9693135Z 2025-12-04T09:46:49.9693274Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9693469Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9693870Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9693979Z res = mod(**inputs) 2025-12-04T09:46:49.9694599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9694725Z outputs = self.mobilebert( 2025-12-04T09:46:49.9695339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9695468Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9696073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9696201Z layer_outputs = layer_module( 2025-12-04T09:46:49.9696809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9697032Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9697645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9697836Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9697851Z 2025-12-04T09:46:49.9697993Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9698176Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9698576Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9698695Z res = mod(**inputs) 2025-12-04T09:46:49.9699305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9699447Z outputs = self.mobilebert( 2025-12-04T09:46:49.9700057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9700187Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9701211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9701513Z layer_outputs = layer_module( 2025-12-04T09:46:49.9702165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9702565Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9703188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9703425Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9704092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9704256Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9704282Z 2025-12-04T09:46:49.9704491Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9704682Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9705105Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9705217Z res = mod(**inputs) 2025-12-04T09:46:49.9705847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9705989Z outputs = self.mobilebert( 2025-12-04T09:46:49.9706612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9706873Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9707500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9707624Z layer_outputs = layer_module( 2025-12-04T09:46:49.9708272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9708587Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9709208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9709457Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9710084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9710332Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9710957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9711121Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9711138Z 2025-12-04T09:46:49.9711291Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9711483Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9711909Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9712026Z res = mod(**inputs) 2025-12-04T09:46:49.9712654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9712791Z outputs = self.mobilebert( 2025-12-04T09:46:49.9713412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9713541Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9714177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9714305Z layer_outputs = layer_module( 2025-12-04T09:46:49.9714935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9715255Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.9715919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.9716133Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.9716757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.9716948Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.9717575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9717806Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9717822Z 2025-12-04T09:46:49.9717976Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9718114Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9718262Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9718396Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9718534Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9718679Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9718923Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9719056Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9719198Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9719331Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9719515Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9719924Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9720034Z res = mod(**inputs) 2025-12-04T09:46:49.9720654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9720773Z outputs = self.mobilebert( 2025-12-04T09:46:49.9721382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9721521Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9722127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9722245Z layer_outputs = layer_module( 2025-12-04T09:46:49.9722855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.9723002Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.9723619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.9723834Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.9724438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.9724677Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9725281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9725453Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9725468Z 2025-12-04T09:46:49.9725602Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9725790Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9726209Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9726323Z res = mod(**inputs) 2025-12-04T09:46:49.9726931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9727067Z outputs = self.mobilebert( 2025-12-04T09:46:49.9727673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9727830Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9728449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9728567Z layer_outputs = layer_module( 2025-12-04T09:46:49.9729214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9729379Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9730032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9730246Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9730848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9731055Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9731070Z 2025-12-04T09:46:49.9731206Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9731387Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9731794Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9731905Z res = mod(**inputs) 2025-12-04T09:46:49.9732509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9732638Z outputs = self.mobilebert( 2025-12-04T09:46:49.9733251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9733381Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9733986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9734106Z layer_outputs = layer_module( 2025-12-04T09:46:49.9734715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9734878Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9735480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9735718Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9736332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9736569Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9737176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9737336Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9737363Z 2025-12-04T09:46:49.9737497Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9737679Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9738088Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9738200Z res = mod(**inputs) 2025-12-04T09:46:49.9738883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9739225Z outputs = self.mobilebert( 2025-12-04T09:46:49.9740033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9740171Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9740792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9740953Z layer_outputs = layer_module( 2025-12-04T09:46:49.9741583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9741751Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9742403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9742611Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9743309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9743548Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9743565Z 2025-12-04T09:46:49.9743706Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9743892Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9744318Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9744429Z res = mod(**inputs) 2025-12-04T09:46:49.9745066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9745195Z outputs = self.mobilebert( 2025-12-04T09:46:49.9745816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9745958Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9746578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9746696Z layer_outputs = layer_module( 2025-12-04T09:46:49.9747463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9747633Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9748257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9748492Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9749114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9749350Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9749974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9750138Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9750154Z 2025-12-04T09:46:49.9750346Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9750530Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9750947Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9751054Z res = mod(**inputs) 2025-12-04T09:46:49.9751674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9751803Z outputs = self.mobilebert( 2025-12-04T09:46:49.9752427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9752563Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9753180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9753300Z layer_outputs = layer_module( 2025-12-04T09:46:49.9753929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9754134Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9754760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9754994Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9755615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9755822Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9755928Z 2025-12-04T09:46:49.9756066Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9756260Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9756675Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9756783Z res = mod(**inputs) 2025-12-04T09:46:49.9757417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9757538Z outputs = self.mobilebert( 2025-12-04T09:46:49.9758159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9758299Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9759012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9759141Z layer_outputs = layer_module( 2025-12-04T09:46:49.9759747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9759906Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9760518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9760750Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9761361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9761591Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9762193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9762357Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9762372Z 2025-12-04T09:46:49.9762504Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9762680Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9763082Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9763190Z res = mod(**inputs) 2025-12-04T09:46:49.9763796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9763914Z outputs = self.mobilebert( 2025-12-04T09:46:49.9764513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9764643Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9765249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9765368Z layer_outputs = layer_module( 2025-12-04T09:46:49.9765974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9766184Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9766789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9767012Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9767027Z 2025-12-04T09:46:49.9767155Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9767341Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9767780Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9767887Z res = mod(**inputs) 2025-12-04T09:46:49.9768548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9768665Z outputs = self.mobilebert( 2025-12-04T09:46:49.9769273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9769395Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9770000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9770125Z layer_outputs = layer_module( 2025-12-04T09:46:49.9770725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9771037Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9771646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9771865Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9772475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9772629Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9772646Z 2025-12-04T09:46:49.9772786Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9772968Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9773363Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9773477Z res = mod(**inputs) 2025-12-04T09:46:49.9774087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9774212Z outputs = self.mobilebert( 2025-12-04T09:46:49.9774823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9774944Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9775551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9775670Z layer_outputs = layer_module( 2025-12-04T09:46:49.9776268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9776575Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9777177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9777408Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9778011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9778228Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9778836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9779019Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9779034Z 2025-12-04T09:46:49.9779169Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9779352Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9779747Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9779886Z res = mod(**inputs) 2025-12-04T09:46:49.9780500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9780617Z outputs = self.mobilebert( 2025-12-04T09:46:49.9781302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9781425Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9782038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9782159Z layer_outputs = layer_module( 2025-12-04T09:46:49.9782757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9783069Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.9783679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.9783883Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.9784494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.9784636Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.9785250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9785407Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9785422Z 2025-12-04T09:46:49.9785561Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9785693Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9785820Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9785958Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9786089Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9786211Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9786352Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9786479Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9786609Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9786835Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9787019Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9787612Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9787723Z res = mod(**inputs) 2025-12-04T09:46:49.9788352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9788494Z outputs = self.mobilebert( 2025-12-04T09:46:49.9789116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9789242Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9789870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9789996Z layer_outputs = layer_module( 2025-12-04T09:46:49.9790627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.9790775Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.9791404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.9791675Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.9792293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.9792562Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9793182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9793393Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9793410Z 2025-12-04T09:46:49.9793553Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9793738Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9794145Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9794260Z res = mod(**inputs) 2025-12-04T09:46:49.9794879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9795008Z outputs = self.mobilebert( 2025-12-04T09:46:49.9795633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9795757Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9796394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9796513Z layer_outputs = layer_module( 2025-12-04T09:46:49.9797136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9797303Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9797919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9798133Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9798754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9798953Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9798976Z 2025-12-04T09:46:49.9799218Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9799393Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9799803Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9799907Z res = mod(**inputs) 2025-12-04T09:46:49.9800509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9800643Z outputs = self.mobilebert( 2025-12-04T09:46:49.9806809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9806955Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9807584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9807712Z layer_outputs = layer_module( 2025-12-04T09:46:49.9808344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9808514Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9809136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9809386Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9810095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9810339Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9810962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9811162Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9811178Z 2025-12-04T09:46:49.9811329Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9811592Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9812017Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9812130Z res = mod(**inputs) 2025-12-04T09:46:49.9812762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9812899Z outputs = self.mobilebert( 2025-12-04T09:46:49.9813527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9813652Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9814290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9814412Z layer_outputs = layer_module( 2025-12-04T09:46:49.9815047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9815215Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9815835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9816046Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9816678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9816885Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9816901Z 2025-12-04T09:46:49.9817042Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9817233Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9817661Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9817773Z res = mod(**inputs) 2025-12-04T09:46:49.9818503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9818632Z outputs = self.mobilebert( 2025-12-04T09:46:49.9819235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9819366Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9819976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9820092Z layer_outputs = layer_module( 2025-12-04T09:46:49.9820707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9820873Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9821487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9821717Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9822313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9822547Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9823191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9823340Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9823362Z 2025-12-04T09:46:49.9823492Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9823700Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9824105Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9824209Z res = mod(**inputs) 2025-12-04T09:46:49.9824865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9824989Z outputs = self.mobilebert( 2025-12-04T09:46:49.9825594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9825726Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9826326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9826438Z layer_outputs = layer_module( 2025-12-04T09:46:49.9827305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9827478Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9828104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9828317Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9828942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9829147Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9829165Z 2025-12-04T09:46:49.9829305Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9829489Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9829916Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9830028Z res = mod(**inputs) 2025-12-04T09:46:49.9830658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9830778Z outputs = self.mobilebert( 2025-12-04T09:46:49.9831409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9831545Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9832163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9832287Z layer_outputs = layer_module( 2025-12-04T09:46:49.9832913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9833080Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9833711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9833949Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9834573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9834809Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9835432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9835632Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9835648Z 2025-12-04T09:46:49.9835783Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9835966Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9836381Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9836519Z res = mod(**inputs) 2025-12-04T09:46:49.9837138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9837265Z outputs = self.mobilebert( 2025-12-04T09:46:49.9837942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9838074Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9838696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9838816Z layer_outputs = layer_module( 2025-12-04T09:46:49.9839539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9839751Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9840362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9840556Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9840571Z 2025-12-04T09:46:49.9840708Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9840898Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9841298Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9841400Z res = mod(**inputs) 2025-12-04T09:46:49.9842012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9842143Z outputs = self.mobilebert( 2025-12-04T09:46:49.9842764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9842893Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9843500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9843624Z layer_outputs = layer_module( 2025-12-04T09:46:49.9844233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9844555Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9845163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9845392Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9846013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9846170Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9846187Z 2025-12-04T09:46:49.9846337Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9846521Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9846927Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9847048Z res = mod(**inputs) 2025-12-04T09:46:49.9847653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9847775Z outputs = self.mobilebert( 2025-12-04T09:46:49.9848391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9848549Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9849168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9849316Z layer_outputs = layer_module( 2025-12-04T09:46:49.9849918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9850282Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9850889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9851125Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9851730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9851958Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9852574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9852738Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9852753Z 2025-12-04T09:46:49.9852904Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9853089Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9853495Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9853620Z res = mod(**inputs) 2025-12-04T09:46:49.9854227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9854347Z outputs = self.mobilebert( 2025-12-04T09:46:49.9854969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9855094Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9855715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9855838Z layer_outputs = layer_module( 2025-12-04T09:46:49.9856443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9856765Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.9857374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.9857569Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.9858193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.9858342Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.9858962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9859124Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9859139Z 2025-12-04T09:46:49.9859273Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9859420Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9859559Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9859703Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9859834Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9859965Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9860108Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9860240Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9860406Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9860545Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9860730Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9861135Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9861282Z res = mod(**inputs) 2025-12-04T09:46:49.9861892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9862027Z outputs = self.mobilebert( 2025-12-04T09:46:49.9862706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9862835Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9863497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9863657Z layer_outputs = layer_module( 2025-12-04T09:46:49.9869429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.9869602Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.9870236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.9870480Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.9871109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.9871350Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9871997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9872163Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9872180Z 2025-12-04T09:46:49.9872335Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9872527Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9872942Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9873073Z res = mod(**inputs) 2025-12-04T09:46:49.9873702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9873843Z outputs = self.mobilebert( 2025-12-04T09:46:49.9874472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9874604Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9875243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9875369Z layer_outputs = layer_module( 2025-12-04T09:46:49.9875996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9876179Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9876810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9877028Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9877662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9877864Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9877880Z 2025-12-04T09:46:49.9878037Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9878230Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9878707Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9878819Z res = mod(**inputs) 2025-12-04T09:46:49.9879548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9879727Z outputs = self.mobilebert( 2025-12-04T09:46:49.9880338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9880467Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9881144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9881269Z layer_outputs = layer_module( 2025-12-04T09:46:49.9881887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9882056Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9882663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9882904Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9883512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9883752Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9884360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9884516Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9884531Z 2025-12-04T09:46:49.9884678Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9884860Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9885261Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9885375Z res = mod(**inputs) 2025-12-04T09:46:49.9885981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9886119Z outputs = self.mobilebert( 2025-12-04T09:46:49.9886725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9886852Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9887463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9887586Z layer_outputs = layer_module( 2025-12-04T09:46:49.9888203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9888364Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9888968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9889176Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9889784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9889980Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9890014Z 2025-12-04T09:46:49.9890151Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9890332Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9890746Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9890852Z res = mod(**inputs) 2025-12-04T09:46:49.9891486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9891619Z outputs = self.mobilebert( 2025-12-04T09:46:49.9892226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9892391Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9893000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9893118Z layer_outputs = layer_module( 2025-12-04T09:46:49.9893784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9893944Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9894545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9894791Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9895397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9895627Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9896232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9896383Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9896403Z 2025-12-04T09:46:49.9896549Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9896731Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9897138Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9897243Z res = mod(**inputs) 2025-12-04T09:46:49.9897848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9897982Z outputs = self.mobilebert( 2025-12-04T09:46:49.9898584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9898708Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9899324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9899449Z layer_outputs = layer_module( 2025-12-04T09:46:49.9900062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9900224Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9901370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9901599Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9902230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9902441Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9902456Z 2025-12-04T09:46:49.9902597Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9902789Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9903220Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9903335Z res = mod(**inputs) 2025-12-04T09:46:49.9903964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9904095Z outputs = self.mobilebert( 2025-12-04T09:46:49.9904804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9904939Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9905564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9905731Z layer_outputs = layer_module( 2025-12-04T09:46:49.9906365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9906606Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9907357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9907597Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9908225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9908475Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9909096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9909276Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9909292Z 2025-12-04T09:46:49.9909431Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9909624Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9910050Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9910162Z res = mod(**inputs) 2025-12-04T09:46:49.9910793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9910932Z outputs = self.mobilebert( 2025-12-04T09:46:49.9911561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9911706Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9912328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9912453Z layer_outputs = layer_module( 2025-12-04T09:46:49.9913090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9913313Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9913937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9914154Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9914173Z 2025-12-04T09:46:49.9914309Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9914506Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9914919Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9915030Z res = mod(**inputs) 2025-12-04T09:46:49.9915665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9915787Z outputs = self.mobilebert( 2025-12-04T09:46:49.9916432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9916564Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9917182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9917317Z layer_outputs = layer_module( 2025-12-04T09:46:49.9917974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9918283Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9919026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9919280Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9919970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9920131Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9920145Z 2025-12-04T09:46:49.9920277Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9920469Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9920867Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9920991Z res = mod(**inputs) 2025-12-04T09:46:49.9921600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9921718Z outputs = self.mobilebert( 2025-12-04T09:46:49.9922336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9922460Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9923066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9923201Z layer_outputs = layer_module( 2025-12-04T09:46:49.9923803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9924117Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9924722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9924948Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9925564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9925792Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9926615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9926955Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9926971Z 2025-12-04T09:46:49.9927109Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9927394Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9927815Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9927942Z res = mod(**inputs) 2025-12-04T09:46:49.9928565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9928691Z outputs = self.mobilebert( 2025-12-04T09:46:49.9929326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9929451Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9930074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9930207Z layer_outputs = layer_module( 2025-12-04T09:46:49.9930885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9931263Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:49.9931889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:49.9932089Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:49.9932757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:49.9932906Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:49.9933599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9933759Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9933774Z 2025-12-04T09:46:49.9933913Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9934061Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9934197Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9934332Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9934473Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9934604Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9934747Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9934879Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9935011Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9935156Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9935346Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9935764Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9935884Z res = mod(**inputs) 2025-12-04T09:46:49.9936510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9936634Z outputs = self.mobilebert( 2025-12-04T09:46:49.9937274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9937398Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9938038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9938164Z layer_outputs = layer_module( 2025-12-04T09:46:49.9938901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:49.9939058Z self_attention_outputs = self.attention( 2025-12-04T09:46:49.9939662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:49.9939891Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:49.9940497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:49.9940728Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9941343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9941503Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9941518Z 2025-12-04T09:46:49.9941659Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9941845Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9942250Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9942366Z res = mod(**inputs) 2025-12-04T09:46:49.9942975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9943134Z outputs = self.mobilebert( 2025-12-04T09:46:49.9943745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9943869Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9944478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9944629Z layer_outputs = layer_module( 2025-12-04T09:46:49.9945236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9945472Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9946079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9946288Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9946991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9947362Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9947377Z 2025-12-04T09:46:49.9947534Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9947733Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9948145Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9948262Z res = mod(**inputs) 2025-12-04T09:46:49.9948900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9949031Z outputs = self.mobilebert( 2025-12-04T09:46:49.9949658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9949788Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9950428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9950550Z layer_outputs = layer_module( 2025-12-04T09:46:49.9951172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9951352Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9951979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9952229Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9952852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9953083Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9953720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9953881Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9953897Z 2025-12-04T09:46:49.9954047Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9954242Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9954665Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9954793Z res = mod(**inputs) 2025-12-04T09:46:49.9955429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9955562Z outputs = self.mobilebert( 2025-12-04T09:46:49.9956190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9956356Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9956985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9957110Z layer_outputs = layer_module( 2025-12-04T09:46:49.9957730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9957940Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9958626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9958842Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9959576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9959769Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9959787Z 2025-12-04T09:46:49.9959935Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9960117Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9960527Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9960633Z res = mod(**inputs) 2025-12-04T09:46:49.9961241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9961377Z outputs = self.mobilebert( 2025-12-04T09:46:49.9961989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9962110Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9962722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9962843Z layer_outputs = layer_module( 2025-12-04T09:46:49.9963457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9963618Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9964215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9964452Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9965055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9965287Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9965894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9966047Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9966064Z 2025-12-04T09:46:49.9966206Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9966390Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9966789Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9966909Z res = mod(**inputs) 2025-12-04T09:46:49.9967515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9967645Z outputs = self.mobilebert( 2025-12-04T09:46:49.9968255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9968380Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9968998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9969149Z layer_outputs = layer_module( 2025-12-04T09:46:49.9969763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9969927Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9970534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:49.9970789Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:49.9971485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9971678Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9971703Z 2025-12-04T09:46:49.9971841Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9972025Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9972434Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9972540Z res = mod(**inputs) 2025-12-04T09:46:49.9973149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9973279Z outputs = self.mobilebert( 2025-12-04T09:46:49.9973888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9974020Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9974624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9974739Z layer_outputs = layer_module( 2025-12-04T09:46:49.9975347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:49.9975513Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:49.9976114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:49.9976351Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:49.9976958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:49.9977194Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9977804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9977961Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9977976Z 2025-12-04T09:46:49.9978121Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9978305Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9978717Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9978827Z res = mod(**inputs) 2025-12-04T09:46:49.9979430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9979565Z outputs = self.mobilebert( 2025-12-04T09:46:49.9980173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9980294Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9980920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9981043Z layer_outputs = layer_module( 2025-12-04T09:46:49.9981655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:49.9981899Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:49.9982499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:49.9982703Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:49.9982746Z 2025-12-04T09:46:49.9982885Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9983078Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9983481Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9983640Z res = mod(**inputs) 2025-12-04T09:46:49.9984264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9984385Z outputs = self.mobilebert( 2025-12-04T09:46:49.9984986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9985121Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9985727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9985854Z layer_outputs = layer_module( 2025-12-04T09:46:49.9986464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9986858Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9987661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:49.9987893Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:49.9988583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9988752Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9988769Z 2025-12-04T09:46:49.9988908Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9989218Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9989641Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9989753Z res = mod(**inputs) 2025-12-04T09:46:49.9990402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9990529Z outputs = self.mobilebert( 2025-12-04T09:46:49.9991162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9991293Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9991915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9992056Z layer_outputs = layer_module( 2025-12-04T09:46:49.9992682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:49.9993009Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:49.9993694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:49.9993935Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:49.9994568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:49.9994800Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:49.9995428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:49.9995639Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:49.9995655Z 2025-12-04T09:46:49.9995793Z cudagraph partition due to non gpu ops 2025-12-04T09:46:49.9995990Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:49.9996433Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:49.9996541Z res = mod(**inputs) 2025-12-04T09:46:49.9997235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:49.9997361Z outputs = self.mobilebert( 2025-12-04T09:46:49.9997994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:49.9998128Z encoder_outputs = self.encoder( 2025-12-04T09:46:49.9998756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:49.9998896Z layer_outputs = layer_module( 2025-12-04T09:46:49.9999513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:49.9999844Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0000471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0000669Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0001705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0001862Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0002492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0002666Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0002681Z 2025-12-04T09:46:50.0002819Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0002972Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0003107Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0003239Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0003384Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0003518Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0003656Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0003798Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0003929Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0004071Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0004256Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0004673Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0004798Z res = mod(**inputs) 2025-12-04T09:46:50.0005430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0005560Z outputs = self.mobilebert( 2025-12-04T09:46:50.0006196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0006320Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0006958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0007083Z layer_outputs = layer_module( 2025-12-04T09:46:50.0007697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0007936Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0008560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0008778Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0009464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0009707Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0010424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0010596Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0010611Z 2025-12-04T09:46:50.0010753Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0010961Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0011384Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0011508Z res = mod(**inputs) 2025-12-04T09:46:50.0012134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0012263Z outputs = self.mobilebert( 2025-12-04T09:46:50.0012902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0013033Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0013761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0013897Z layer_outputs = layer_module( 2025-12-04T09:46:50.0014504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0014688Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0015293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0015491Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0016110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0016306Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0016321Z 2025-12-04T09:46:50.0016476Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0016662Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0017068Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0017196Z res = mod(**inputs) 2025-12-04T09:46:50.0017802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0017926Z outputs = self.mobilebert( 2025-12-04T09:46:50.0018542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0018671Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0019290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0019410Z layer_outputs = layer_module( 2025-12-04T09:46:50.0020020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0020201Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0020806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0021095Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0021702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0021930Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0022580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0022741Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0022756Z 2025-12-04T09:46:50.0022956Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0023142Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0023542Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0023666Z res = mod(**inputs) 2025-12-04T09:46:50.0024278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0024404Z outputs = self.mobilebert( 2025-12-04T09:46:50.0025027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0025162Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0025783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0025904Z layer_outputs = layer_module( 2025-12-04T09:46:50.0026512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0026692Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0027573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0027794Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0028425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0028622Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0028654Z 2025-12-04T09:46:50.0028803Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0028988Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0029406Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0029528Z res = mod(**inputs) 2025-12-04T09:46:50.0030153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0030292Z outputs = self.mobilebert( 2025-12-04T09:46:50.0030919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0031055Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0031692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0031820Z layer_outputs = layer_module( 2025-12-04T09:46:50.0032457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0032629Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0033253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0033500Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0034122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0034400Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0035037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0035228Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0035243Z 2025-12-04T09:46:50.0035393Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0035581Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0036048Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0036172Z res = mod(**inputs) 2025-12-04T09:46:50.0036795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0036934Z outputs = self.mobilebert( 2025-12-04T09:46:50.0037563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0037694Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0038330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0038457Z layer_outputs = layer_module( 2025-12-04T09:46:50.0039189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0039372Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0039974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0040186Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0040790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0040989Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0041004Z 2025-12-04T09:46:50.0041154Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0041336Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0041756Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0041866Z res = mod(**inputs) 2025-12-04T09:46:50.0042480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0042616Z outputs = self.mobilebert( 2025-12-04T09:46:50.0043223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0043350Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0043969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0044089Z layer_outputs = layer_module( 2025-12-04T09:46:50.0044708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0044878Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0045483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0045732Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0046340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0046574Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0047179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0047368Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0047383Z 2025-12-04T09:46:50.0047530Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0047715Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0048141Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0048260Z res = mod(**inputs) 2025-12-04T09:46:50.0048922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0049056Z outputs = self.mobilebert( 2025-12-04T09:46:50.0049663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0049791Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0050410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0050531Z layer_outputs = layer_module( 2025-12-04T09:46:50.0051220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0056515Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0057151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0057368Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0057384Z 2025-12-04T09:46:50.0057533Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0057725Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0058153Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0058268Z res = mod(**inputs) 2025-12-04T09:46:50.0058915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0059041Z outputs = self.mobilebert( 2025-12-04T09:46:50.0059667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0059814Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0060445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0060583Z layer_outputs = layer_module( 2025-12-04T09:46:50.0061202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0061517Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0062158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0062389Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0063022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0063188Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0063203Z 2025-12-04T09:46:50.0063340Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0063541Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0064070Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0064177Z res = mod(**inputs) 2025-12-04T09:46:50.0064801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0064969Z outputs = self.mobilebert( 2025-12-04T09:46:50.0065591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0065716Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0066354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0066493Z layer_outputs = layer_module( 2025-12-04T09:46:50.0067448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0067887Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0068527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0068763Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0069403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0069635Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0070262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0070436Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0070452Z 2025-12-04T09:46:50.0070597Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0070800Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0071215Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0071322Z res = mod(**inputs) 2025-12-04T09:46:50.0071963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0072092Z outputs = self.mobilebert( 2025-12-04T09:46:50.0072725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0072854Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0073482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0073620Z layer_outputs = layer_module( 2025-12-04T09:46:50.0074246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:50.0074564Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0075201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0075401Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0076031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0076180Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0076806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0076973Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0076988Z 2025-12-04T09:46:50.0077131Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0077279Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0077415Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0077550Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0077696Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0077862Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0077997Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0078147Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0078280Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0078524Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0078715Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0079147Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0079268Z res = mod(**inputs) 2025-12-04T09:46:50.0079935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0080060Z outputs = self.mobilebert( 2025-12-04T09:46:50.0080680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0080808Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0081419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0081556Z layer_outputs = layer_module( 2025-12-04T09:46:50.0082165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0082329Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0082935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0083158Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0083780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0084014Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0084636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0084792Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0084807Z 2025-12-04T09:46:50.0084943Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0085138Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0085542Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0085650Z res = mod(**inputs) 2025-12-04T09:46:50.0086276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0086400Z outputs = self.mobilebert( 2025-12-04T09:46:50.0087026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0087151Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0087758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0087894Z layer_outputs = layer_module( 2025-12-04T09:46:50.0088503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0088685Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0089297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0089499Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0090127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0090321Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0090368Z 2025-12-04T09:46:50.0090514Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0090698Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0091095Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0091219Z res = mod(**inputs) 2025-12-04T09:46:50.0091859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0091984Z outputs = self.mobilebert( 2025-12-04T09:46:50.0092662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0092794Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0093416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0093538Z layer_outputs = layer_module( 2025-12-04T09:46:50.0094147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0094322Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0094931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0095167Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0095794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0096025Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0096648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0096802Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0096819Z 2025-12-04T09:46:50.0096951Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0097143Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0097546Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0097659Z res = mod(**inputs) 2025-12-04T09:46:50.0098270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0098395Z outputs = self.mobilebert( 2025-12-04T09:46:50.0099015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0099141Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0099757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0099877Z layer_outputs = layer_module( 2025-12-04T09:46:50.0100482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0100653Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0101632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0102017Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0102663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0102866Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0102884Z 2025-12-04T09:46:50.0103217Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0103413Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0103829Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0104035Z res = mod(**inputs) 2025-12-04T09:46:50.0104668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0104809Z outputs = self.mobilebert( 2025-12-04T09:46:50.0105479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0105610Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0106320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0106445Z layer_outputs = layer_module( 2025-12-04T09:46:50.0107170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0107349Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0107976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0108225Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0108845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0109081Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0109725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0109888Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0109903Z 2025-12-04T09:46:50.0110058Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0110247Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0110664Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0110789Z res = mod(**inputs) 2025-12-04T09:46:50.0111414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0111540Z outputs = self.mobilebert( 2025-12-04T09:46:50.0112182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0112313Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0112955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0113080Z layer_outputs = layer_module( 2025-12-04T09:46:50.0113767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0118757Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0139043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0139350Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0140004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0140314Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0140331Z 2025-12-04T09:46:50.0140518Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0140775Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0141228Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0141330Z res = mod(**inputs) 2025-12-04T09:46:50.0141947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0142169Z outputs = self.mobilebert( 2025-12-04T09:46:50.0142779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0142902Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0143540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0143661Z layer_outputs = layer_module( 2025-12-04T09:46:50.0144350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0144517Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0145124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0145353Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0145964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0146188Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0146903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0147064Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0147079Z 2025-12-04T09:46:50.0147390Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0147588Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0148002Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0148109Z res = mod(**inputs) 2025-12-04T09:46:50.0148740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0148868Z outputs = self.mobilebert( 2025-12-04T09:46:50.0149499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0149631Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0150252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0150381Z layer_outputs = layer_module( 2025-12-04T09:46:50.0151004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0151229Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0151848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0152050Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0152066Z 2025-12-04T09:46:50.0152212Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0152395Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0152801Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0152918Z res = mod(**inputs) 2025-12-04T09:46:50.0153540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0153673Z outputs = self.mobilebert( 2025-12-04T09:46:50.0154294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0154416Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0155049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0155209Z layer_outputs = layer_module( 2025-12-04T09:46:50.0155833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0156140Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0156794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0157081Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0157699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0157853Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0157875Z 2025-12-04T09:46:50.0158008Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0158198Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0158608Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0158712Z res = mod(**inputs) 2025-12-04T09:46:50.0159438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0159565Z outputs = self.mobilebert( 2025-12-04T09:46:50.0160171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0160303Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0160904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0161019Z layer_outputs = layer_module( 2025-12-04T09:46:50.0161622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0161925Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0162524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0162758Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0163359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0163585Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0164189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0164338Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0164354Z 2025-12-04T09:46:50.0164492Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0164673Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0165076Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0165177Z res = mod(**inputs) 2025-12-04T09:46:50.0165783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0165910Z outputs = self.mobilebert( 2025-12-04T09:46:50.0166518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0166638Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0167244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0167358Z layer_outputs = layer_module( 2025-12-04T09:46:50.0167989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:50.0168291Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0168893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0169117Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0169772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0169922Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0170521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0170671Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0170689Z 2025-12-04T09:46:50.0170826Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0170954Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0171079Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0171214Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0171341Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0171473Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0171599Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0171723Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0172316Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0172448Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0172628Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0173037Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0173141Z res = mod(**inputs) 2025-12-04T09:46:50.0173756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0173875Z outputs = self.mobilebert( 2025-12-04T09:46:50.0174475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0174607Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0175202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0175315Z layer_outputs = layer_module( 2025-12-04T09:46:50.0175966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0176132Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0177158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0177385Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0178009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0178251Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0178878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0179041Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0179057Z 2025-12-04T09:46:50.0179197Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0179384Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0179805Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0179909Z res = mod(**inputs) 2025-12-04T09:46:50.0180525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0180704Z outputs = self.mobilebert( 2025-12-04T09:46:50.0181402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0181579Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0182200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0182317Z layer_outputs = layer_module( 2025-12-04T09:46:50.0183459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0183623Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0184251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0184455Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0185078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0185281Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0185300Z 2025-12-04T09:46:50.0185435Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0185631Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0186047Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0186149Z res = mod(**inputs) 2025-12-04T09:46:50.0186884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0187008Z outputs = self.mobilebert( 2025-12-04T09:46:50.0187626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0187768Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0188389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0188522Z layer_outputs = layer_module( 2025-12-04T09:46:50.0189142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0189303Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0189934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0190165Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0190791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0191024Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0191651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0191815Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0191833Z 2025-12-04T09:46:50.0191967Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0192151Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0192571Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0192676Z res = mod(**inputs) 2025-12-04T09:46:50.0193302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0193421Z outputs = self.mobilebert( 2025-12-04T09:46:50.0194036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0194207Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0194829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0194999Z layer_outputs = layer_module( 2025-12-04T09:46:50.0195616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0195778Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0196457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0196658Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0197276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0197486Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0197501Z 2025-12-04T09:46:50.0197635Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0197823Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0198236Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0198450Z res = mod(**inputs) 2025-12-04T09:46:50.0199064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0199178Z outputs = self.mobilebert( 2025-12-04T09:46:50.0199789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0199910Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0200511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0200643Z layer_outputs = layer_module( 2025-12-04T09:46:50.0201832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0202012Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0202644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0202885Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0203515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0203751Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0204368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0204538Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0204554Z 2025-12-04T09:46:50.0204692Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0204882Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0205295Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0205400Z res = mod(**inputs) 2025-12-04T09:46:50.0206031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0206155Z outputs = self.mobilebert( 2025-12-04T09:46:50.0206771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0206911Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0207624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0207752Z layer_outputs = layer_module( 2025-12-04T09:46:50.0208374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0208623Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0209255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0209528Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0210158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0210353Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0210368Z 2025-12-04T09:46:50.0210504Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0210691Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0211099Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0211203Z res = mod(**inputs) 2025-12-04T09:46:50.0211826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0211952Z outputs = self.mobilebert( 2025-12-04T09:46:50.0212586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0212712Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0213332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0213460Z layer_outputs = layer_module( 2025-12-04T09:46:50.0214177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0214345Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0214944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0215168Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0215772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0215998Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0216601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0216759Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0216774Z 2025-12-04T09:46:50.0216906Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0217093Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0217486Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0217586Z res = mod(**inputs) 2025-12-04T09:46:50.0218195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0218321Z outputs = self.mobilebert( 2025-12-04T09:46:50.0218931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0219053Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0219655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0219782Z layer_outputs = layer_module( 2025-12-04T09:46:50.0220419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0220627Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0221238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0221458Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0221473Z 2025-12-04T09:46:50.0221607Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0221840Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0222236Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0222344Z res = mod(**inputs) 2025-12-04T09:46:50.0222944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0223069Z outputs = self.mobilebert( 2025-12-04T09:46:50.0223668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0223786Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0224390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0224506Z layer_outputs = layer_module( 2025-12-04T09:46:50.0225108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0225411Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0226006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0226232Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0226918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0227069Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0227084Z 2025-12-04T09:46:50.0227391Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0227578Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0227994Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0228100Z res = mod(**inputs) 2025-12-04T09:46:50.0228722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0228853Z outputs = self.mobilebert( 2025-12-04T09:46:50.0229469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0229594Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0230222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0230339Z layer_outputs = layer_module( 2025-12-04T09:46:50.0230959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0231271Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0231892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0232125Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0232742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0233011Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0233635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0233789Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0233834Z 2025-12-04T09:46:50.0233972Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0234159Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0234575Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0234734Z res = mod(**inputs) 2025-12-04T09:46:50.0235353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0235480Z outputs = self.mobilebert( 2025-12-04T09:46:50.0236099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0236226Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0236853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0236974Z layer_outputs = layer_module( 2025-12-04T09:46:50.0237605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:50.0237921Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0238625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0238843Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0239577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0239744Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0240372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0240532Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0240551Z 2025-12-04T09:46:50.0240705Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0240843Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0240978Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0241134Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0241273Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0241421Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0241556Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0241689Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0241837Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0241970Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0242160Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0242591Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0242701Z res = mod(**inputs) 2025-12-04T09:46:50.0243398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0243538Z outputs = self.mobilebert( 2025-12-04T09:46:50.0244171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0244314Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0244937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0245062Z layer_outputs = layer_module( 2025-12-04T09:46:50.0245762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0245912Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0246551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0246808Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0247438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0247742Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0248370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0248545Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0248564Z 2025-12-04T09:46:50.0248700Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0248891Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0249318Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0249428Z res = mod(**inputs) 2025-12-04T09:46:50.0250056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0250195Z outputs = self.mobilebert( 2025-12-04T09:46:50.0250929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0251069Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0251672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0251794Z layer_outputs = layer_module( 2025-12-04T09:46:50.0252410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0252573Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0253172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0253385Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0253993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0254194Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0254209Z 2025-12-04T09:46:50.0254344Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0254528Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0254944Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0255054Z res = mod(**inputs) 2025-12-04T09:46:50.0255675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0255797Z outputs = self.mobilebert( 2025-12-04T09:46:50.0256403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0256541Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0257150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0257271Z layer_outputs = layer_module( 2025-12-04T09:46:50.0257888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0258055Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0258704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0258936Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0259540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0259810Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0260467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0260641Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0260656Z 2025-12-04T09:46:50.0260793Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0260979Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0261399Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0261510Z res = mod(**inputs) 2025-12-04T09:46:50.0262129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0262250Z outputs = self.mobilebert( 2025-12-04T09:46:50.0262857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0262997Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0263604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0263725Z layer_outputs = layer_module( 2025-12-04T09:46:50.0264343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0264508Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0265128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0265326Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0265931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0266141Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0266156Z 2025-12-04T09:46:50.0266295Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0266499Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0266991Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0267268Z res = mod(**inputs) 2025-12-04T09:46:50.0267914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0268047Z outputs = self.mobilebert( 2025-12-04T09:46:50.0268665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0268804Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0269432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0269564Z layer_outputs = layer_module( 2025-12-04T09:46:50.0270195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0270361Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0270990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0271226Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0271894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0272126Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0272746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0272948Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0272964Z 2025-12-04T09:46:50.0273104Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0273341Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0273764Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0273873Z res = mod(**inputs) 2025-12-04T09:46:50.0274508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0274634Z outputs = self.mobilebert( 2025-12-04T09:46:50.0275256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0275396Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0276022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0276149Z layer_outputs = layer_module( 2025-12-04T09:46:50.0276772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0276938Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0277567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0277774Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0278397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0278602Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0278620Z 2025-12-04T09:46:50.0278756Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0278953Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0279467Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0279575Z res = mod(**inputs) 2025-12-04T09:46:50.0280192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0280313Z outputs = self.mobilebert( 2025-12-04T09:46:50.0280924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0281048Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0281650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0281780Z layer_outputs = layer_module( 2025-12-04T09:46:50.0282381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0282537Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0283151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0283381Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0283987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0284241Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0284838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0284998Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0285041Z 2025-12-04T09:46:50.0285176Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0285360Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0285758Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0285929Z res = mod(**inputs) 2025-12-04T09:46:50.0286545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0286662Z outputs = self.mobilebert( 2025-12-04T09:46:50.0287260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0287395Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0287993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0288117Z layer_outputs = layer_module( 2025-12-04T09:46:50.0288718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0288930Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0289546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0289740Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0289755Z 2025-12-04T09:46:50.0289895Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0290079Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0290476Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0290590Z res = mod(**inputs) 2025-12-04T09:46:50.0291193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0291315Z outputs = self.mobilebert( 2025-12-04T09:46:50.0291934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0292060Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0292672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0292789Z layer_outputs = layer_module( 2025-12-04T09:46:50.0293387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0293700Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0294307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0294539Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0295141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0295298Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0295312Z 2025-12-04T09:46:50.0295457Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0295640Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0296038Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0296190Z res = mod(**inputs) 2025-12-04T09:46:50.0296795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0296921Z outputs = self.mobilebert( 2025-12-04T09:46:50.0297522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0297677Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0298287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0298460Z layer_outputs = layer_module( 2025-12-04T09:46:50.0299075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0299375Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0299978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0300210Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0306368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0306631Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0307346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0307514Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0307531Z 2025-12-04T09:46:50.0307684Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0307876Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0308293Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0308419Z res = mod(**inputs) 2025-12-04T09:46:50.0309043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0309173Z outputs = self.mobilebert( 2025-12-04T09:46:50.0309792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0309920Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0310553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0310677Z layer_outputs = layer_module( 2025-12-04T09:46:50.0311308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:50.0311623Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0312250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0312450Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0313080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0313228Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0313864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0314023Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0314038Z 2025-12-04T09:46:50.0314184Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0314316Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0314448Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0314679Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0314815Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0314944Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0315088Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0315218Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0315356Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0315529Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0315716Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0316143Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0316329Z res = mod(**inputs) 2025-12-04T09:46:50.0316953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0317085Z outputs = self.mobilebert( 2025-12-04T09:46:50.0317703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0317838Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0318458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0318574Z layer_outputs = layer_module( 2025-12-04T09:46:50.0319304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0319445Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0320045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0320273Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0320871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0321112Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0321719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0321867Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0321885Z 2025-12-04T09:46:50.0322029Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0322211Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0322625Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0322729Z res = mod(**inputs) 2025-12-04T09:46:50.0323328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0323460Z outputs = self.mobilebert( 2025-12-04T09:46:50.0324065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0324250Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0324871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0324988Z layer_outputs = layer_module( 2025-12-04T09:46:50.0325599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0325760Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0326363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0326570Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0327176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0327410Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0327425Z 2025-12-04T09:46:50.0327557Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0327736Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0328141Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0328276Z res = mod(**inputs) 2025-12-04T09:46:50.0328873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0329063Z outputs = self.mobilebert( 2025-12-04T09:46:50.0329660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0329788Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0330388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0330505Z layer_outputs = layer_module( 2025-12-04T09:46:50.0331119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0331283Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0331898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0332127Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0332728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0332963Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0333570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0333733Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0333747Z 2025-12-04T09:46:50.0333878Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0334059Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0334472Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0334576Z res = mod(**inputs) 2025-12-04T09:46:50.0335184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0335315Z outputs = self.mobilebert( 2025-12-04T09:46:50.0335917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0336049Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0336654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0336772Z layer_outputs = layer_module( 2025-12-04T09:46:50.0337380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0337543Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0338147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0338349Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0338950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0339147Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0339162Z 2025-12-04T09:46:50.0339292Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0339503Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0339916Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0340023Z res = mod(**inputs) 2025-12-04T09:46:50.0340638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0340788Z outputs = self.mobilebert( 2025-12-04T09:46:50.0341384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0341583Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0342190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0342319Z layer_outputs = layer_module( 2025-12-04T09:46:50.0342921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0343083Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0343699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0343931Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0344532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0344772Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0345379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0345537Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0345551Z 2025-12-04T09:46:50.0345687Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0345866Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0346273Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0346379Z res = mod(**inputs) 2025-12-04T09:46:50.0347104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0347401Z outputs = self.mobilebert( 2025-12-04T09:46:50.0348025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0348169Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0348791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0348910Z layer_outputs = layer_module( 2025-12-04T09:46:50.0349537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0349707Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0350335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0350539Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0351164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0351374Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0351390Z 2025-12-04T09:46:50.0351527Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0351723Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0352134Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0352280Z res = mod(**inputs) 2025-12-04T09:46:50.0352910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0353032Z outputs = self.mobilebert( 2025-12-04T09:46:50.0353651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0353816Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0354497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0354633Z layer_outputs = layer_module( 2025-12-04T09:46:50.0355253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0355412Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0356041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0356275Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0356896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0357129Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0357752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0357917Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0357932Z 2025-12-04T09:46:50.0358067Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0358248Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0358671Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0358778Z res = mod(**inputs) 2025-12-04T09:46:50.0359509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0359627Z outputs = self.mobilebert( 2025-12-04T09:46:50.0360229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0360363Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0360968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0361099Z layer_outputs = layer_module( 2025-12-04T09:46:50.0361701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0361911Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0362524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0362713Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0362727Z 2025-12-04T09:46:50.0362870Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0363054Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0363521Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0363638Z res = mod(**inputs) 2025-12-04T09:46:50.0369404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0369530Z outputs = self.mobilebert( 2025-12-04T09:46:50.0370167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0370342Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0370971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0371092Z layer_outputs = layer_module( 2025-12-04T09:46:50.0371711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0372062Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0372746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0372974Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0373603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0373764Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0373779Z 2025-12-04T09:46:50.0373924Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0374110Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0374515Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0374639Z res = mod(**inputs) 2025-12-04T09:46:50.0375264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0375393Z outputs = self.mobilebert( 2025-12-04T09:46:50.0376025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0376156Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0376783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0376908Z layer_outputs = layer_module( 2025-12-04T09:46:50.0377527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0377850Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0378477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0378718Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0379343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0379570Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0380317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0380474Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0380489Z 2025-12-04T09:46:50.0380627Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0380807Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0381206Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0381325Z res = mod(**inputs) 2025-12-04T09:46:50.0381932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0382064Z outputs = self.mobilebert( 2025-12-04T09:46:50.0382667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0382794Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0383404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0383549Z layer_outputs = layer_module( 2025-12-04T09:46:50.0384148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:50.0384458Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0385095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0385292Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0385943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0386088Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0386786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0386960Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0386976Z 2025-12-04T09:46:50.0387283Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0387417Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0387551Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0387814Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0387949Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0388080Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0388222Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0388357Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0388484Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0388623Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0388807Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0389228Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0389342Z res = mod(**inputs) 2025-12-04T09:46:50.0389967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0390101Z outputs = self.mobilebert( 2025-12-04T09:46:50.0390722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0390852Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0391488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0391608Z layer_outputs = layer_module( 2025-12-04T09:46:50.0392245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0392389Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0393008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0393240Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0393855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0394101Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0394733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0394890Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0394905Z 2025-12-04T09:46:50.0395056Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0395245Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0395651Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0395806Z res = mod(**inputs) 2025-12-04T09:46:50.0396424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0396554Z outputs = self.mobilebert( 2025-12-04T09:46:50.0397183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0397355Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0398045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0398164Z layer_outputs = layer_module( 2025-12-04T09:46:50.0398793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0398963Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0399679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0399886Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0400486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0400692Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0400707Z 2025-12-04T09:46:50.0401404Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0401602Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0402020Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0402133Z res = mod(**inputs) 2025-12-04T09:46:50.0402762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0402899Z outputs = self.mobilebert( 2025-12-04T09:46:50.0403530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0403665Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0404293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0404417Z layer_outputs = layer_module( 2025-12-04T09:46:50.0405046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0405213Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0405832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0406076Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0406699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0406938Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0407561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0407715Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0407731Z 2025-12-04T09:46:50.0407884Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0408080Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0408504Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0408616Z res = mod(**inputs) 2025-12-04T09:46:50.0409238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0409456Z outputs = self.mobilebert( 2025-12-04T09:46:50.0410087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0410217Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0410898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0411025Z layer_outputs = layer_module( 2025-12-04T09:46:50.0411729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0411901Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0412524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0412742Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0413372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0413691Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0413706Z 2025-12-04T09:46:50.0413844Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0414027Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0414448Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0414558Z res = mod(**inputs) 2025-12-04T09:46:50.0415176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0415297Z outputs = self.mobilebert( 2025-12-04T09:46:50.0415911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0416049Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0416655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0416775Z layer_outputs = layer_module( 2025-12-04T09:46:50.0417392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0417552Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0418166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0418398Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0419005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0419242Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0419846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0420011Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0420029Z 2025-12-04T09:46:50.0420163Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0420345Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0420754Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0420867Z res = mod(**inputs) 2025-12-04T09:46:50.0421474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0421611Z outputs = self.mobilebert( 2025-12-04T09:46:50.0422218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0422421Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0423028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0423148Z layer_outputs = layer_module( 2025-12-04T09:46:50.0423792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0423955Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0424626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0424827Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0425432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0425641Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0425655Z 2025-12-04T09:46:50.0425792Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0426043Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0426817Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0426965Z res = mod(**inputs) 2025-12-04T09:46:50.0427606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0427731Z outputs = self.mobilebert( 2025-12-04T09:46:50.0428359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0428500Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0429126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0429266Z layer_outputs = layer_module( 2025-12-04T09:46:50.0429887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0430056Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0430701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0430995Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0431620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0431862Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0432488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0432657Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0432672Z 2025-12-04T09:46:50.0432812Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0433000Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0433416Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0433532Z res = mod(**inputs) 2025-12-04T09:46:50.0434167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0434292Z outputs = self.mobilebert( 2025-12-04T09:46:50.0434916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0435057Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0435682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0435849Z layer_outputs = layer_module( 2025-12-04T09:46:50.0436478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0436701Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0437363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0437564Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0437636Z 2025-12-04T09:46:50.0437777Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0437977Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0438392Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0438512Z res = mod(**inputs) 2025-12-04T09:46:50.0439240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0439363Z outputs = self.mobilebert( 2025-12-04T09:46:50.0439981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0440109Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0440718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0440849Z layer_outputs = layer_module( 2025-12-04T09:46:50.0441457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0441771Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0442378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0442606Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0443226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0443388Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0443403Z 2025-12-04T09:46:50.0443551Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0443735Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0444139Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0444256Z res = mod(**inputs) 2025-12-04T09:46:50.0444860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0444979Z outputs = self.mobilebert( 2025-12-04T09:46:50.0445599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0445727Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0446333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0446453Z layer_outputs = layer_module( 2025-12-04T09:46:50.0447056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0447370Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0447976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0448211Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0448859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0449083Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0449698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0449885Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0449900Z 2025-12-04T09:46:50.0450052Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0450293Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0450697Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0450814Z res = mod(**inputs) 2025-12-04T09:46:50.0451420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0451543Z outputs = self.mobilebert( 2025-12-04T09:46:50.0452162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0452288Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0452902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0453024Z layer_outputs = layer_module( 2025-12-04T09:46:50.0453632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:50.0453958Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0454564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0454768Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0455378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0455525Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0456141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0456297Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0456312Z 2025-12-04T09:46:50.0456443Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0456596Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0456721Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0456862Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0456989Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0457119Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0457259Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0457391Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0457521Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0457661Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0457843Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0458243Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0458364Z res = mod(**inputs) 2025-12-04T09:46:50.0458974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0459113Z outputs = self.mobilebert( 2025-12-04T09:46:50.0459718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0459842Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0460458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0460609Z layer_outputs = layer_module( 2025-12-04T09:46:50.0461222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0461369Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0462002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0462229Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0462888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0463112Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0463722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0463878Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0463893Z 2025-12-04T09:46:50.0464035Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0464216Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0464613Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0464736Z res = mod(**inputs) 2025-12-04T09:46:50.0465337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0465475Z outputs = self.mobilebert( 2025-12-04T09:46:50.0466081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0466208Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0466941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0467070Z layer_outputs = layer_module( 2025-12-04T09:46:50.0467861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0468042Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0468670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0468889Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0469519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0469721Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0469737Z 2025-12-04T09:46:50.0469888Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0470083Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0470504Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0470616Z res = mod(**inputs) 2025-12-04T09:46:50.0471242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0471381Z outputs = self.mobilebert( 2025-12-04T09:46:50.0472009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0472142Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0472783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0472909Z layer_outputs = layer_module( 2025-12-04T09:46:50.0473544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0473756Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0474379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0474659Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0475282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0475584Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0476212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0476374Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0476389Z 2025-12-04T09:46:50.0476543Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0476735Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0477157Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0477269Z res = mod(**inputs) 2025-12-04T09:46:50.0477899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0478040Z outputs = self.mobilebert( 2025-12-04T09:46:50.0478666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0478804Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0479547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0479671Z layer_outputs = layer_module( 2025-12-04T09:46:50.0480287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0480458Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0481062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0481277Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0481891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0482095Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0482110Z 2025-12-04T09:46:50.0482242Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0482428Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0482841Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0482953Z res = mod(**inputs) 2025-12-04T09:46:50.0483559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0483690Z outputs = self.mobilebert( 2025-12-04T09:46:50.0484297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0484436Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0485038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0485164Z layer_outputs = layer_module( 2025-12-04T09:46:50.0485777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0485938Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0486537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0486807Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0487407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0487679Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0488286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0488569Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0488595Z 2025-12-04T09:46:50.0488734Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0489107Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0489716Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0489830Z res = mod(**inputs) 2025-12-04T09:46:50.0490460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0490601Z outputs = self.mobilebert( 2025-12-04T09:46:50.0491225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0491373Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0492004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0492129Z layer_outputs = layer_module( 2025-12-04T09:46:50.0492768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0492936Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0493630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0493853Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0494482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0494690Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0494706Z 2025-12-04T09:46:50.0494848Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0495036Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0495467Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0495578Z res = mod(**inputs) 2025-12-04T09:46:50.0496215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0496344Z outputs = self.mobilebert( 2025-12-04T09:46:50.0496972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0497115Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0497744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0497875Z layer_outputs = layer_module( 2025-12-04T09:46:50.0498511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0498684Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0499307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0499540Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0500199Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0500438Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0501469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0501720Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0501736Z 2025-12-04T09:46:50.0501877Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0502139Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0502570Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0502679Z res = mod(**inputs) 2025-12-04T09:46:50.0503306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0503448Z outputs = self.mobilebert( 2025-12-04T09:46:50.0504073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0504213Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0504836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0504961Z layer_outputs = layer_module( 2025-12-04T09:46:50.0505604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0505826Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0506463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0506665Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0506683Z 2025-12-04T09:46:50.0506925Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0507131Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0507545Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0507654Z res = mod(**inputs) 2025-12-04T09:46:50.0508300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0508427Z outputs = self.mobilebert( 2025-12-04T09:46:50.0509067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0509199Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0509824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0509966Z layer_outputs = layer_module( 2025-12-04T09:46:50.0510593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0510923Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0511545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0511784Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0512419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0512579Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0512594Z 2025-12-04T09:46:50.0512744Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0512933Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0513393Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0513511Z res = mod(**inputs) 2025-12-04T09:46:50.0514131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0514282Z outputs = self.mobilebert( 2025-12-04T09:46:50.0514914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0515044Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0515733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0515861Z layer_outputs = layer_module( 2025-12-04T09:46:50.0516486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0516816Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0517437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0517669Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0518314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0518653Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0519275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0519433Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0519448Z 2025-12-04T09:46:50.0519583Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0519779Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0520182Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0520295Z res = mod(**inputs) 2025-12-04T09:46:50.0520901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0521025Z outputs = self.mobilebert( 2025-12-04T09:46:50.0521641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0521771Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0522377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0522512Z layer_outputs = layer_module( 2025-12-04T09:46:50.0523116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:50.0523441Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0524051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0524245Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0524864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0525012Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0525638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0525793Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0525808Z 2025-12-04T09:46:50.0525944Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0526117Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0526253Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0526385Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0526523Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0526653Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0526818Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0526948Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0527078Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0527218Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0527456Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0527864Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0528005Z res = mod(**inputs) 2025-12-04T09:46:50.0528619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0528763Z outputs = self.mobilebert( 2025-12-04T09:46:50.0529376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0529507Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0530128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0530255Z layer_outputs = layer_module( 2025-12-04T09:46:50.0530865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0531026Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0531630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0531864Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0532473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0532698Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0533316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0533471Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0533485Z 2025-12-04T09:46:50.0533627Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0533813Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0534213Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0534334Z res = mod(**inputs) 2025-12-04T09:46:50.0534946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0535070Z outputs = self.mobilebert( 2025-12-04T09:46:50.0535693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0535817Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0536436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0536560Z layer_outputs = layer_module( 2025-12-04T09:46:50.0537169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0537421Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0538310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0538550Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0539229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0539517Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0539532Z 2025-12-04T09:46:50.0539705Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0540034Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0540494Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0540637Z res = mod(**inputs) 2025-12-04T09:46:50.0541416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0541588Z outputs = self.mobilebert( 2025-12-04T09:46:50.0542265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0542470Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0543132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0543339Z layer_outputs = layer_module( 2025-12-04T09:46:50.0543990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0544241Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0544863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0545164Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0545879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0546143Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0546928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0547348Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0547364Z 2025-12-04T09:46:50.0547523Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0547861Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0548315Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0548476Z res = mod(**inputs) 2025-12-04T09:46:50.0549201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0549369Z outputs = self.mobilebert( 2025-12-04T09:46:50.0550092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0550283Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0551039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0551268Z layer_outputs = layer_module( 2025-12-04T09:46:50.0556851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0557097Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0557834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0558098Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0558826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0559115Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0559132Z 2025-12-04T09:46:50.0559361Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0559565Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0560056Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0560320Z res = mod(**inputs) 2025-12-04T09:46:50.0560991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0561155Z outputs = self.mobilebert( 2025-12-04T09:46:50.0561927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0562074Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0562858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0563024Z layer_outputs = layer_module( 2025-12-04T09:46:50.0563684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0563943Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0564608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0564981Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0565665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0565938Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0566792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0566991Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0567006Z 2025-12-04T09:46:50.0567212Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0567466Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0567926Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0568122Z res = mod(**inputs) 2025-12-04T09:46:50.0568764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0568923Z outputs = self.mobilebert( 2025-12-04T09:46:50.0569609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0569804Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0570515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0570676Z layer_outputs = layer_module( 2025-12-04T09:46:50.0571323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0571583Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0572204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0572537Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0573183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0573415Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0573431Z 2025-12-04T09:46:50.0573658Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0573878Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0574443Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0574604Z res = mod(**inputs) 2025-12-04T09:46:50.0575247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0575493Z outputs = self.mobilebert( 2025-12-04T09:46:50.0576135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0576324Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0577060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0577238Z layer_outputs = layer_module( 2025-12-04T09:46:50.0577940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0578145Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0578786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0579079Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0579762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0580102Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0580750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0580943Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0581008Z 2025-12-04T09:46:50.0581181Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0581381Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0581936Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0582081Z res = mod(**inputs) 2025-12-04T09:46:50.0582730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0582968Z outputs = self.mobilebert( 2025-12-04T09:46:50.0583612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0583844Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0584513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0584672Z layer_outputs = layer_module( 2025-12-04T09:46:50.0585361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0585616Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0586285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0586562Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0586580Z 2025-12-04T09:46:50.0586875Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0587321Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0587785Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0587936Z res = mod(**inputs) 2025-12-04T09:46:50.0588632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0588853Z outputs = self.mobilebert( 2025-12-04T09:46:50.0589630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0589797Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0590466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0590722Z layer_outputs = layer_module( 2025-12-04T09:46:50.0591364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0591897Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0592590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0592860Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0593585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0593787Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0593803Z 2025-12-04T09:46:50.0594047Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0594295Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0594753Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0594963Z res = mod(**inputs) 2025-12-04T09:46:50.0595632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0595795Z outputs = self.mobilebert( 2025-12-04T09:46:50.0596515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0596702Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0597428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0597593Z layer_outputs = layer_module( 2025-12-04T09:46:50.0598255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0598640Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0599439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0599773Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0600425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0600689Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0601957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0602141Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0602160Z 2025-12-04T09:46:50.0602491Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0602737Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0603191Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0603390Z res = mod(**inputs) 2025-12-04T09:46:50.0604059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0604281Z outputs = self.mobilebert( 2025-12-04T09:46:50.0604961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0605224Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0605938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0606102Z layer_outputs = layer_module( 2025-12-04T09:46:50.0606763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:50.0607228Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0607996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0608288Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0608957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0609148Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0609854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0610090Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0610106Z 2025-12-04T09:46:50.0610349Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0610527Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0610702Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0610940Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0611119Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0611399Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0611571Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0611742Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0611963Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0612146Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0612354Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0612918Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0613069Z res = mod(**inputs) 2025-12-04T09:46:50.0613819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0618968Z outputs = self.mobilebert( 2025-12-04T09:46:50.0619642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0619897Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0620580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0620743Z layer_outputs = layer_module( 2025-12-04T09:46:50.0621455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0621656Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0622346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0622653Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0623336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0623661Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0624343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0624625Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0624642Z 2025-12-04T09:46:50.0624813Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0625152Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0625678Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0625841Z res = mod(**inputs) 2025-12-04T09:46:50.0626557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0626860Z outputs = self.mobilebert( 2025-12-04T09:46:50.0627507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0627878Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0628543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0628706Z layer_outputs = layer_module( 2025-12-04T09:46:50.0629415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0629626Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0630351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0630623Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0631286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0631574Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0631595Z 2025-12-04T09:46:50.0631774Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0632029Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0632517Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0632695Z res = mod(**inputs) 2025-12-04T09:46:50.0633418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0633609Z outputs = self.mobilebert( 2025-12-04T09:46:50.0634319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0634468Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0635168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0635408Z layer_outputs = layer_module( 2025-12-04T09:46:50.0636067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0636274Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0636983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0637239Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0638021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0638298Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0639054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0639299Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0639314Z 2025-12-04T09:46:50.0639487Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0639776Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0640235Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0640423Z res = mod(**inputs) 2025-12-04T09:46:50.0641117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0641275Z outputs = self.mobilebert( 2025-12-04T09:46:50.0641949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0642173Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0642838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0643127Z layer_outputs = layer_module( 2025-12-04T09:46:50.0643785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0644033Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0644652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0644926Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0645642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0645885Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0645900Z 2025-12-04T09:46:50.0646121Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0646343Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0646766Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0647017Z res = mod(**inputs) 2025-12-04T09:46:50.0647659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0647827Z outputs = self.mobilebert( 2025-12-04T09:46:50.0648522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0648685Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0649392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0649572Z layer_outputs = layer_module( 2025-12-04T09:46:50.0650224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0650477Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0651119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0651409Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0652087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0652400Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0653103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0653298Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0653313Z 2025-12-04T09:46:50.0653533Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0653733Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0654215Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0654437Z res = mod(**inputs) 2025-12-04T09:46:50.0655112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0655304Z outputs = self.mobilebert( 2025-12-04T09:46:50.0655997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0656138Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0656914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0657104Z layer_outputs = layer_module( 2025-12-04T09:46:50.0657746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0658050Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0658701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0658996Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0659663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0659896Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0659912Z 2025-12-04T09:46:50.0660136Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0660358Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0660856Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0661033Z res = mod(**inputs) 2025-12-04T09:46:50.0661697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0661906Z outputs = self.mobilebert( 2025-12-04T09:46:50.0662550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0662725Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0663398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0669197Z layer_outputs = layer_module( 2025-12-04T09:46:50.0669911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0670108Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0670745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0670992Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0671636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0671866Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0672485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0672662Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0672679Z 2025-12-04T09:46:50.0672823Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0673030Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0673445Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0673552Z res = mod(**inputs) 2025-12-04T09:46:50.0674192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0674318Z outputs = self.mobilebert( 2025-12-04T09:46:50.0674955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0675167Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0675813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0676009Z layer_outputs = layer_module( 2025-12-04T09:46:50.0676766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0677047Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0677755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0677959Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0677975Z 2025-12-04T09:46:50.0678131Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0678319Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0678738Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0678870Z res = mod(**inputs) 2025-12-04T09:46:50.0679494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0679638Z outputs = self.mobilebert( 2025-12-04T09:46:50.0680268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0680394Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0681107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0681229Z layer_outputs = layer_module( 2025-12-04T09:46:50.0681853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0682176Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0682800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0683041Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0683666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0683825Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0683841Z 2025-12-04T09:46:50.0683995Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0684180Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0684606Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0684715Z res = mod(**inputs) 2025-12-04T09:46:50.0685339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0685481Z outputs = self.mobilebert( 2025-12-04T09:46:50.0686105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0686228Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0686861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0686982Z layer_outputs = layer_module( 2025-12-04T09:46:50.0687618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0687927Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0688654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0688923Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0689519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0689755Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0690388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0690539Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0690554Z 2025-12-04T09:46:50.0690803Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0690986Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0691399Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0691508Z res = mod(**inputs) 2025-12-04T09:46:50.0692121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0692253Z outputs = self.mobilebert( 2025-12-04T09:46:50.0692860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0692987Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0693601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0693715Z layer_outputs = layer_module( 2025-12-04T09:46:50.0694334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:50.0694643Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0695246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0695449Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0696054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0696211Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0696815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0696969Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0696988Z 2025-12-04T09:46:50.0697133Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0697260Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0697387Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0697524Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0697650Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0697793Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0697922Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0698051Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0698195Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0698324Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0698511Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0698927Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0699032Z res = mod(**inputs) 2025-12-04T09:46:50.0699644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0699779Z outputs = self.mobilebert( 2025-12-04T09:46:50.0700383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0700550Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0701558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0701874Z layer_outputs = layer_module( 2025-12-04T09:46:50.0702517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0702744Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0703970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0704202Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0704824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0705072Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0705699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0705860Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0705895Z 2025-12-04T09:46:50.0706035Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0706226Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0706659Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0706875Z res = mod(**inputs) 2025-12-04T09:46:50.0707507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0707648Z outputs = self.mobilebert( 2025-12-04T09:46:50.0708271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0708413Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0709040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0709160Z layer_outputs = layer_module( 2025-12-04T09:46:50.0709797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0709968Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0710590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0710805Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0711425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0711635Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0711654Z 2025-12-04T09:46:50.0711794Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0711984Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0712409Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0712518Z res = mod(**inputs) 2025-12-04T09:46:50.0713159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0713283Z outputs = self.mobilebert( 2025-12-04T09:46:50.0713909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0714054Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0714677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0714849Z layer_outputs = layer_module( 2025-12-04T09:46:50.0715488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0715655Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0716288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0716553Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0717233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0717480Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0718101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0718272Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0718287Z 2025-12-04T09:46:50.0718535Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0718719Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0719132Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0719240Z res = mod(**inputs) 2025-12-04T09:46:50.0719847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0719983Z outputs = self.mobilebert( 2025-12-04T09:46:50.0720591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0720732Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0721335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0721452Z layer_outputs = layer_module( 2025-12-04T09:46:50.0722069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0722228Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0722841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0723037Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0723644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0723851Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0723866Z 2025-12-04T09:46:50.0723997Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0724185Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0724585Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0724687Z res = mod(**inputs) 2025-12-04T09:46:50.0725298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0725417Z outputs = self.mobilebert( 2025-12-04T09:46:50.0726015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0726149Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0726751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0726880Z layer_outputs = layer_module( 2025-12-04T09:46:50.0727483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0727673Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0728289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0728514Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0729157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0729379Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0730033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0730197Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0730212Z 2025-12-04T09:46:50.0730342Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0730525Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0730946Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0731051Z res = mod(**inputs) 2025-12-04T09:46:50.0731669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0731795Z outputs = self.mobilebert( 2025-12-04T09:46:50.0732397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0732536Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0733143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0733277Z layer_outputs = layer_module( 2025-12-04T09:46:50.0733882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0734044Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0734656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0734850Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0735459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0735666Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0735681Z 2025-12-04T09:46:50.0735820Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0736011Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0736415Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0736520Z res = mod(**inputs) 2025-12-04T09:46:50.0737147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0737277Z outputs = self.mobilebert( 2025-12-04T09:46:50.0737901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0738031Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0738709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0739022Z layer_outputs = layer_module( 2025-12-04T09:46:50.0739832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0740000Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0740637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0740915Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0741551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0741782Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0742433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0742605Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0742678Z 2025-12-04T09:46:50.0742822Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0743022Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0747033Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0747193Z res = mod(**inputs) 2025-12-04T09:46:50.0747840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0747981Z outputs = self.mobilebert( 2025-12-04T09:46:50.0748616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0748757Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0749385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0749511Z layer_outputs = layer_module( 2025-12-04T09:46:50.0750153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0750372Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0751025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0751240Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0751256Z 2025-12-04T09:46:50.0751395Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0751594Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0752010Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0752119Z res = mod(**inputs) 2025-12-04T09:46:50.0752756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0752883Z outputs = self.mobilebert( 2025-12-04T09:46:50.0753522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0753654Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0754278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0754416Z layer_outputs = layer_module( 2025-12-04T09:46:50.0755036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0755346Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0755983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0756212Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0756840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0757001Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0757017Z 2025-12-04T09:46:50.0757218Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0757422Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0757835Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0757961Z res = mod(**inputs) 2025-12-04T09:46:50.0758693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0758863Z outputs = self.mobilebert( 2025-12-04T09:46:50.0759513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0759635Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0760235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0760442Z layer_outputs = layer_module( 2025-12-04T09:46:50.0761050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0761366Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0761969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0762196Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0762818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0763043Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0763658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0763815Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0763833Z 2025-12-04T09:46:50.0763968Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0764165Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0764559Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0764665Z res = mod(**inputs) 2025-12-04T09:46:50.0765281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0765396Z outputs = self.mobilebert( 2025-12-04T09:46:50.0766018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0766141Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0766743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0766875Z layer_outputs = layer_module( 2025-12-04T09:46:50.0767475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 496, in forward 2025-12-04T09:46:50.0767793Z query_tensor, key_tensor, value_tensor, layer_input = self.bottleneck(hidden_states) 2025-12-04T09:46:50.0768399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 444, in forward 2025-12-04T09:46:50.0768595Z shared_attention_input = self.attention(hidden_states) 2025-12-04T09:46:50.0769213Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 410, in forward 2025-12-04T09:46:50.0769358Z layer_input = self.LayerNorm(layer_input) 2025-12-04T09:46:50.0769975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0770132Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0770179Z 2025-12-04T09:46:50.0770315Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0770460Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0770591Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0770723Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0770894Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0771023Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0771152Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0771292Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0771452Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0771594Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0771774Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0772216Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0772336Z res = mod(**inputs) 2025-12-04T09:46:50.0772944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0773067Z outputs = self.mobilebert( 2025-12-04T09:46:50.0773685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0773810Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0774425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0774545Z layer_outputs = layer_module( 2025-12-04T09:46:50.0775148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 500, in forward 2025-12-04T09:46:50.0775302Z self_attention_outputs = self.attention( 2025-12-04T09:46:50.0775910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 344, in forward 2025-12-04T09:46:50.0776140Z attention_output = self.output(self_outputs[0], layer_input) 2025-12-04T09:46:50.0776745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 295, in forward 2025-12-04T09:46:50.0776972Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0777592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0777752Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0777770Z 2025-12-04T09:46:50.0777907Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0778102Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0778506Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0778627Z res = mod(**inputs) 2025-12-04T09:46:50.0779234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0779357Z outputs = self.mobilebert( 2025-12-04T09:46:50.0779979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0780108Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0780723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0780846Z layer_outputs = layer_module( 2025-12-04T09:46:50.0781451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0781630Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0782233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0782461Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0783077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0783324Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0783339Z 2025-12-04T09:46:50.0783485Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0783670Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0784098Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0784219Z res = mod(**inputs) 2025-12-04T09:46:50.0784822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0784989Z outputs = self.mobilebert( 2025-12-04T09:46:50.0785595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0785718Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0786332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0786451Z layer_outputs = layer_module( 2025-12-04T09:46:50.0787168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0787349Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0788136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0788393Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0789018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0789252Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0789889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0790052Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0790068Z 2025-12-04T09:46:50.0790222Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0790412Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0790829Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0790956Z res = mod(**inputs) 2025-12-04T09:46:50.0791583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0791711Z outputs = self.mobilebert( 2025-12-04T09:46:50.0792349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0792480Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0793118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0793242Z layer_outputs = layer_module( 2025-12-04T09:46:50.0793869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0794052Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0794674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0794896Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0795522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0795760Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0795776Z 2025-12-04T09:46:50.0795927Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0796116Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0796558Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0796684Z res = mod(**inputs) 2025-12-04T09:46:50.0797335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0797472Z outputs = self.mobilebert( 2025-12-04T09:46:50.0798127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0798255Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0798902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0799023Z layer_outputs = layer_module( 2025-12-04T09:46:50.0799660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0799831Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0800458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0800714Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0806820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0807068Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0807714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0807872Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0807891Z 2025-12-04T09:46:50.0808047Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0808245Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0808661Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0808779Z res = mod(**inputs) 2025-12-04T09:46:50.0809413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0809552Z outputs = self.mobilebert( 2025-12-04T09:46:50.0810182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0810312Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0810952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0811075Z layer_outputs = layer_module( 2025-12-04T09:46:50.0811704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0811873Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0812493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 469, in forward 2025-12-04T09:46:50.0812709Z intermediate_output = self.intermediate(hidden_states) 2025-12-04T09:46:50.0813444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0813651Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0813772Z 2025-12-04T09:46:50.0813909Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0814095Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0814507Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0814618Z res = mod(**inputs) 2025-12-04T09:46:50.0815264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0815395Z outputs = self.mobilebert( 2025-12-04T09:46:50.0816041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0816179Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0816836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0816957Z layer_outputs = layer_module( 2025-12-04T09:46:50.0817576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 515, in forward 2025-12-04T09:46:50.0817741Z attention_output = ffn_module(attention_output) 2025-12-04T09:46:50.0818344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 470, in forward 2025-12-04T09:46:50.0818588Z layer_outputs = self.output(intermediate_output, hidden_states) 2025-12-04T09:46:50.0819193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 458, in forward 2025-12-04T09:46:50.0819426Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0820034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0820186Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0820204Z 2025-12-04T09:46:50.0820347Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0820530Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0820945Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0821055Z res = mod(**inputs) 2025-12-04T09:46:50.0821662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0821791Z outputs = self.mobilebert( 2025-12-04T09:46:50.0822400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0822528Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0823146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0823269Z layer_outputs = layer_module( 2025-12-04T09:46:50.0823881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 518, in forward 2025-12-04T09:46:50.0824096Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:46:50.0824698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 360, in forward 2025-12-04T09:46:50.0824906Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:46:50.0824921Z 2025-12-04T09:46:50.0825057Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0825253Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0825652Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0825757Z res = mod(**inputs) 2025-12-04T09:46:50.0826375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0826557Z outputs = self.mobilebert( 2025-12-04T09:46:50.0827444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0827589Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0828253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0828391Z layer_outputs = layer_module( 2025-12-04T09:46:50.0829045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0829363Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0830024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 397, in forward 2025-12-04T09:46:50.0830254Z layer_output = self.LayerNorm(layer_output + residual_tensor_1) 2025-12-04T09:46:50.0830884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0831040Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0831059Z 2025-12-04T09:46:50.0831196Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0831387Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0831800Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0831918Z res = mod(**inputs) 2025-12-04T09:46:50.0832543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 976, in forward 2025-12-04T09:46:50.0832664Z outputs = self.mobilebert( 2025-12-04T09:46:50.0833298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 794, in forward 2025-12-04T09:46:50.0833433Z encoder_outputs = self.encoder( 2025-12-04T09:46:50.0834054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 557, in forward 2025-12-04T09:46:50.0834180Z layer_outputs = layer_module( 2025-12-04T09:46:50.0834797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 519, in forward 2025-12-04T09:46:50.0835118Z layer_output = self.output(intermediate_output, attention_output, hidden_states) 2025-12-04T09:46:50.0835745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 398, in forward 2025-12-04T09:46:50.0835977Z layer_output = self.bottleneck(layer_output, residual_tensor_2) 2025-12-04T09:46:50.0836608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 374, in forward 2025-12-04T09:46:50.0836842Z layer_outputs = self.LayerNorm(layer_outputs + residual_tensor) 2025-12-04T09:46:50.0837466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 138, in forward 2025-12-04T09:46:50.0837624Z return input_tensor * self.weight + self.bias 2025-12-04T09:46:50.0837642Z 2025-12-04T09:46:50.0837781Z cudagraph partition due to non gpu ops 2025-12-04T09:46:50.0837978Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0838397Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0838510Z res = mod(**inputs) 2025-12-04T09:46:50.0839252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 989, in forward 2025-12-04T09:46:50.0839416Z prediction_scores = self.cls(sequence_output) 2025-12-04T09:46:50.0840058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 643, in forward 2025-12-04T09:46:50.0840251Z prediction_scores = self.predictions(sequence_output) 2025-12-04T09:46:50.0840853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 632, in forward 2025-12-04T09:46:50.0841289Z hidden_states = hidden_states.matmul(torch.cat([self.decoder.weight.t(), self.dense.weight], dim=0)) 2025-12-04T09:46:50.0841304Z 2025-12-04T09:46:50.0841488Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0841922Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0842029Z res = mod(**inputs) 2025-12-04T09:46:50.0842668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 994, in forward 2025-12-04T09:46:50.0843036Z masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:46:50.0843051Z 2025-12-04T09:46:50.0843235Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:46:50.0843638Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:46:50.0843750Z res = mod(**inputs) 2025-12-04T09:46:50.0844357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 989, in forward 2025-12-04T09:46:50.0844520Z prediction_scores = self.cls(sequence_output) 2025-12-04T09:46:50.0845126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 643, in forward 2025-12-04T09:46:50.0845318Z prediction_scores = self.predictions(sequence_output) 2025-12-04T09:46:50.0845932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 633, in forward 2025-12-04T09:46:50.0846062Z hidden_states += self.decoder.bias 2025-12-04T09:46:50.0846077Z 2025-12-04T09:47:02.0670896Z Compilation time (from dynamo_timed): 62.698574449 2025-12-04T09:47:02.0671386Z pass 2025-12-04T09:47:02.0671907Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:02.0673667Z TIMING: _recursive_pre_grad_passes:0.20035 _recursive_joint_graph_passes:2.20405 _recursive_post_grad_passes:0.24401 async_compile.wait:1.11937 code_gen:9.58874 inductor_compile:18.76105 backend_compile:46.18957 gc:0.00043 entire_frame_compile:62.69857 total_wall_time:62.69857 2025-12-04T09:47:02.0675647Z STATS: call_* op count: 1449 | FakeTensorMode.__torch_dispatch__:76974 | FakeTensor.__torch_dispatch__:11426 | ProxyTorchDispatchMode.__torch_dispatch__:10397 2025-12-04T09:47:02.0676722Z Dynamo produced 1 graphs covering 1449 ops with 0 graph breaks (0 unique) 2025-12-04T09:47:05.9067397Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:47:05.9069119Z import pynvml # type: ignore[import] 2025-12-04T09:47:10.9900218Z 2025-12-04T09:47:14.0703563Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:47:14.0703982Z loading model: 0it [00:03, ?it/s] 2025-12-04T09:47:14.0717500Z cpu eval OPTForCausalLM 2025-12-04T09:47:16.8604158Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:18.1108960Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:19.4214607Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:34.2535816Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2536678Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2537078Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2537459Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2537856Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2538261Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2538741Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2539119Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2539505Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2539894Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2540329Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2540718Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2541111Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2541534Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2541909Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2542368Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2542855Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2547918Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2548325Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2548786Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2549530Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2550226Z res = mod(**inputs) 2025-12-04T09:47:34.2550916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2551683Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2552472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2553303Z outputs = self.model.decoder( 2025-12-04T09:47:34.2554035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2554777Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2555579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2556390Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2557127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2557888Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2558700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2559504Z return func(*args, **kwargs) 2025-12-04T09:47:34.2560275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2561233Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2562068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2562856Z return func(*args, **kwargs) 2025-12-04T09:47:34.2563604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2564418Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2565360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2566366Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2566743Z 2025-12-04T09:47:34.2566905Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2567289Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2567739Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2568472Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2569190Z res = mod(**inputs) 2025-12-04T09:47:34.2569848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2570591Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2571376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2572202Z outputs = self.model.decoder( 2025-12-04T09:47:34.2572907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2573679Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2574447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2575244Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2576003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2576757Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2577526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2578310Z return func(*args, **kwargs) 2025-12-04T09:47:34.2579062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2579894Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2580198Z 2025-12-04T09:47:34.2580337Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2580733Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2581116Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2581612Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2581996Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2582385Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2582758Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2583138Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2583575Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2584299Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2584945Z res = mod(**inputs) 2025-12-04T09:47:34.2585597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2586333Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2587227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2588213Z outputs = self.model.decoder( 2025-12-04T09:47:34.2588937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2589697Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2590487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2591299Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2592027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2592804Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2593591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2594393Z return func(*args, **kwargs) 2025-12-04T09:47:34.2595166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2596010Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2596861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2597730Z return func(*args, **kwargs) 2025-12-04T09:47:34.2598500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2599341Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2600303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2601895Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2602281Z 2025-12-04T09:47:34.2602527Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2602915Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2603368Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2604111Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2604825Z res = mod(**inputs) 2025-12-04T09:47:34.2605574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2606423Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2607210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2608033Z outputs = self.model.decoder( 2025-12-04T09:47:34.2608751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2609501Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2610284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2611097Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2611820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2612587Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2613379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2614177Z return func(*args, **kwargs) 2025-12-04T09:47:34.2614943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2615791Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2616116Z 2025-12-04T09:47:34.2616257Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2616658Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2617066Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2617449Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2617847Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2618238Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2618616Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2619012Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2619456Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2620176Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2620842Z res = mod(**inputs) 2025-12-04T09:47:34.2621646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2622372Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2623129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2623924Z outputs = self.model.decoder( 2025-12-04T09:47:34.2624619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2625328Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2626106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2627062Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2627960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2628717Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2629583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2630383Z return func(*args, **kwargs) 2025-12-04T09:47:34.2631171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2632037Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2632934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2633742Z return func(*args, **kwargs) 2025-12-04T09:47:34.2634500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2635361Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2636321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2637360Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2637743Z 2025-12-04T09:47:34.2637888Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2638297Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2638866Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2639577Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2640229Z res = mod(**inputs) 2025-12-04T09:47:34.2640887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2641628Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2642385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2643176Z outputs = self.model.decoder( 2025-12-04T09:47:34.2643875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2644583Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2645354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2646139Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2646841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2647569Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2648345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2649119Z return func(*args, **kwargs) 2025-12-04T09:47:34.2649845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2650676Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2650992Z 2025-12-04T09:47:34.2651131Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2651522Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2651892Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2652276Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2652658Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2653026Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2653406Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2653790Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2654212Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2654967Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2655619Z res = mod(**inputs) 2025-12-04T09:47:34.2656266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2657015Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2657796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2658582Z outputs = self.model.decoder( 2025-12-04T09:47:34.2659292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2660019Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2660820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2661609Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2662300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2663044Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2663822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2664597Z return func(*args, **kwargs) 2025-12-04T09:47:34.2665322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2666154Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2667087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2668208Z return func(*args, **kwargs) 2025-12-04T09:47:34.2668988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2669847Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2670805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2671816Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2672211Z 2025-12-04T09:47:34.2672355Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2672784Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2673250Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2673989Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2674667Z res = mod(**inputs) 2025-12-04T09:47:34.2675339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2676076Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2676873Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2677685Z outputs = self.model.decoder( 2025-12-04T09:47:34.2678386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2679138Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2679934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2680836Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2681524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2682264Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2683046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2683882Z return func(*args, **kwargs) 2025-12-04T09:47:34.2684612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2685447Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2685746Z 2025-12-04T09:47:34.2685943Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2686316Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2686698Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2687085Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2687506Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2687874Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2688254Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2688644Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2689123Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2689848Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2690513Z res = mod(**inputs) 2025-12-04T09:47:34.2691146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2691881Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2692662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2693454Z outputs = self.model.decoder( 2025-12-04T09:47:34.2694143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2694875Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2695653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2696432Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2697145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2697897Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2698681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2699447Z return func(*args, **kwargs) 2025-12-04T09:47:34.2700197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2701596Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2702451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2703242Z return func(*args, **kwargs) 2025-12-04T09:47:34.2704014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2704879Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2705828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2706975Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2707382Z 2025-12-04T09:47:34.2707526Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2707930Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2708367Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2709112Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2709785Z res = mod(**inputs) 2025-12-04T09:47:34.2710435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2711187Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2711977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2712880Z outputs = self.model.decoder( 2025-12-04T09:47:34.2713582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2714332Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2715179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2715975Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2716751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2717525Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2718488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2719262Z return func(*args, **kwargs) 2025-12-04T09:47:34.2720005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2720838Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2721140Z 2025-12-04T09:47:34.2721292Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2721672Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2722059Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2722442Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2722812Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2723201Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2723587Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2723956Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2724393Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2725117Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2725773Z res = mod(**inputs) 2025-12-04T09:47:34.2726408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2727136Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2727907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2728679Z outputs = self.model.decoder( 2025-12-04T09:47:34.2729376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2730162Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2736085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2736885Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2737618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2738393Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2739188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2739995Z return func(*args, **kwargs) 2025-12-04T09:47:34.2740759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2741625Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2742458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2743259Z return func(*args, **kwargs) 2025-12-04T09:47:34.2744019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2744860Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2745879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2747036Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2747420Z 2025-12-04T09:47:34.2747581Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2748009Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2748456Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2749198Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2749911Z res = mod(**inputs) 2025-12-04T09:47:34.2750570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2751323Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2752156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2752959Z outputs = self.model.decoder( 2025-12-04T09:47:34.2753682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2754430Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2755227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2756028Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2756753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2757519Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2758305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2759195Z return func(*args, **kwargs) 2025-12-04T09:47:34.2759941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2760771Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2761070Z 2025-12-04T09:47:34.2761206Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2761596Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2761984Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2762356Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2762740Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2763124Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2763497Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2763885Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2764322Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2765044Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2765682Z res = mod(**inputs) 2025-12-04T09:47:34.2766339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2767076Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2767837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2768635Z outputs = self.model.decoder( 2025-12-04T09:47:34.2769335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2770135Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2770899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2771687Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2772400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2773139Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2773959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2774745Z return func(*args, **kwargs) 2025-12-04T09:47:34.2775488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2776346Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2777163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2777970Z return func(*args, **kwargs) 2025-12-04T09:47:34.2778717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2779526Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2780485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2781486Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2781855Z 2025-12-04T09:47:34.2781993Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2782378Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2782819Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2783534Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2784171Z res = mod(**inputs) 2025-12-04T09:47:34.2784823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2785554Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2786319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2787399Z outputs = self.model.decoder( 2025-12-04T09:47:34.2788127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2788875Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2789657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2790466Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2791189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2791943Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2792828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2798528Z return func(*args, **kwargs) 2025-12-04T09:47:34.2799514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2810877Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2811285Z 2025-12-04T09:47:34.2811451Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2811868Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2812280Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2812688Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2813189Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2813576Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2813960Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2814327Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2814780Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2815514Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2816161Z res = mod(**inputs) 2025-12-04T09:47:34.2816831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2817696Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2818484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2819264Z outputs = self.model.decoder( 2025-12-04T09:47:34.2819968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2820754Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2821557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2822347Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2823059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2823870Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2824639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2825426Z return func(*args, **kwargs) 2025-12-04T09:47:34.2826178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2827299Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2828164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2828966Z return func(*args, **kwargs) 2025-12-04T09:47:34.2829745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2830585Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2831545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2832576Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2832954Z 2025-12-04T09:47:34.2833113Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2833500Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2833951Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2834696Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2835354Z res = mod(**inputs) 2025-12-04T09:47:34.2836024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2836783Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2837587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2838383Z outputs = self.model.decoder( 2025-12-04T09:47:34.2839217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2839954Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2840709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2841503Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2842218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2842969Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2843736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2844517Z return func(*args, **kwargs) 2025-12-04T09:47:34.2845264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2846086Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2846451Z 2025-12-04T09:47:34.2846590Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2846979Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2847371Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2847742Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2848135Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2848550Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2848917Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2849299Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2849738Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2850478Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2851134Z res = mod(**inputs) 2025-12-04T09:47:34.2851819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2852551Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2853322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2854116Z outputs = self.model.decoder( 2025-12-04T09:47:34.2854822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2855770Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2856749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2857569Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2858305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2859059Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2859871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2860681Z return func(*args, **kwargs) 2025-12-04T09:47:34.2861449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2862293Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2863146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2863947Z return func(*args, **kwargs) 2025-12-04T09:47:34.2864705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2865561Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2866524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2867665Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2868049Z 2025-12-04T09:47:34.2868196Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2868597Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2869047Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2869778Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2870454Z res = mod(**inputs) 2025-12-04T09:47:34.2871126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2871878Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2872669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2873488Z outputs = self.model.decoder( 2025-12-04T09:47:34.2874215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2875024Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2875809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2876623Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2877355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2878146Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2879044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2879882Z return func(*args, **kwargs) 2025-12-04T09:47:34.2880628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2881447Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2881795Z 2025-12-04T09:47:34.2881938Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2882334Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2882707Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2883099Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2883486Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2883857Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2884249Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2884633Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2885141Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2885863Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2886521Z res = mod(**inputs) 2025-12-04T09:47:34.2887160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2887898Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2888676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2889460Z outputs = self.model.decoder( 2025-12-04T09:47:34.2890163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2890895Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2891677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2892444Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2893157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2893904Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2894668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2895449Z return func(*args, **kwargs) 2025-12-04T09:47:34.2896195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2897026Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2897833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2898607Z return func(*args, **kwargs) 2025-12-04T09:47:34.2899345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2900172Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2901671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2902716Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2903101Z 2025-12-04T09:47:34.2903262Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2903753Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2904207Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2904955Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2905627Z res = mod(**inputs) 2025-12-04T09:47:34.2906331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2907209Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2908076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2908877Z outputs = self.model.decoder( 2025-12-04T09:47:34.2909598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2910396Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2911195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2911990Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2912718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2913488Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2914286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2915087Z return func(*args, **kwargs) 2025-12-04T09:47:34.2915858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2916721Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2917031Z 2025-12-04T09:47:34.2917175Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2917635Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2918045Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2918542Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2918923Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2919316Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2919713Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2920114Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2920552Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2921294Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2921973Z res = mod(**inputs) 2025-12-04T09:47:34.2922699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2923447Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2924248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2925063Z outputs = self.model.decoder( 2025-12-04T09:47:34.2925765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2926515Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2927315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2928115Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2928842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2929612Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2930506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2931277Z return func(*args, **kwargs) 2025-12-04T09:47:34.2932021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2932919Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2933727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2934509Z return func(*args, **kwargs) 2025-12-04T09:47:34.2935292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2936120Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2937060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2938065Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2938446Z 2025-12-04T09:47:34.2938617Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2939010Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2939443Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2940168Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2940829Z res = mod(**inputs) 2025-12-04T09:47:34.2941464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2942208Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2942989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2943790Z outputs = self.model.decoder( 2025-12-04T09:47:34.2944474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2945208Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2945990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2946852Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2947754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2948519Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2949324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2950110Z return func(*args, **kwargs) 2025-12-04T09:47:34.2950879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2951730Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2952038Z 2025-12-04T09:47:34.2952180Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2952576Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2952972Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2953366Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2953743Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2954132Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2954522Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2954902Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2955350Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2956098Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2956757Z res = mod(**inputs) 2025-12-04T09:47:34.2957429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2958179Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2958977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2959875Z outputs = self.model.decoder( 2025-12-04T09:47:34.2960614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2961343Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2962099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2962923Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2963629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2964379Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2965175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2965958Z return func(*args, **kwargs) 2025-12-04T09:47:34.2966734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 255, in forward 2025-12-04T09:47:34.2967564Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:47:34.2968365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2969135Z return func(*args, **kwargs) 2025-12-04T09:47:34.2969879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 179, in forward 2025-12-04T09:47:34.2970694Z attn_output, attn_weights = attention_interface( 2025-12-04T09:47:34.2971620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:47:34.2972614Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:47:34.2972981Z 2025-12-04T09:47:34.2973134Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2973508Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2973949Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2974672Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2975303Z res = mod(**inputs) 2025-12-04T09:47:34.2975952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2976681Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2977452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 818, in forward 2025-12-04T09:47:34.2978227Z outputs = self.model.decoder( 2025-12-04T09:47:34.2978923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2979644Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2980477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 648, in forward 2025-12-04T09:47:34.2986487Z layer_outputs = decoder_layer( 2025-12-04T09:47:34.2987346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:47:34.2988116Z return super().__call__(*args, **kwargs) 2025-12-04T09:47:34.2988906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:47:34.2989702Z return func(*args, **kwargs) 2025-12-04T09:47:34.2990463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 282, in forward 2025-12-04T09:47:34.2991321Z hidden_states = self.activation_fn(hidden_states) 2025-12-04T09:47:34.2991626Z 2025-12-04T09:47:34.2991765Z cudagraph partition due to non gpu ops 2025-12-04T09:47:34.2992212Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2992956Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2993684Z res = mod(**inputs) 2025-12-04T09:47:34.2994350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.2995109Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.2995910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 833, in forward 2025-12-04T09:47:34.2996773Z logits = self.lm_head(outputs[0]).contiguous() 2025-12-04T09:47:34.2997080Z 2025-12-04T09:47:34.2997273Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:47:34.2998038Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:47:34.2998801Z res = mod(**inputs) 2025-12-04T09:47:34.2999448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/generic.py", line 918, in wrapper 2025-12-04T09:47:34.3000209Z output = func(self, *args, **kwargs) 2025-12-04T09:47:34.3001360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/opt/modeling_opt.py", line 839, in forward 2025-12-04T09:47:34.3002331Z loss = self.loss_function( 2025-12-04T09:47:34.3003081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:47:34.3004099Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:47:34.3005150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:47:34.3006224Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:47:34.3006772Z 2025-12-04T09:47:43.3473544Z Compilation time (from dynamo_timed): 21.08903745 2025-12-04T09:47:43.4154310Z pass 2025-12-04T09:47:43.4155370Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:43.4160337Z TIMING: _recursive_pre_grad_passes:0.05171 _recursive_joint_graph_passes:0.86734 _recursive_post_grad_passes:0.11183 async_compile.wait:1.19179 code_gen:7.12587 inductor_compile:11.14259 backend_compile:17.42466 gc:0.00039 entire_frame_compile:21.08904 total_wall_time:21.08904 2025-12-04T09:47:43.4163993Z STATS: call_* op count: 379 | FakeTensorMode.__torch_dispatch__:17830 | FakeTensor.__torch_dispatch__:3248 | ProxyTorchDispatchMode.__torch_dispatch__:2777 2025-12-04T09:47:43.4166120Z Dynamo produced 1 graphs covering 379 ops with 0 graph breaks (0 unique) 2025-12-04T09:47:46.3847038Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:47:46.3848839Z import pynvml # type: ignore[import] 2025-12-04T09:47:51.4294073Z 2025-12-04T09:47:53.6038720Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:47:53.6039291Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:47:53.6055013Z cpu eval PLBartForCausalLM 2025-12-04T09:47:54.7708478Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:55.2472100Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:47:55.7062719Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:05.2406464Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2406979Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2407371Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2407774Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2408193Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2408596Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2408985Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2409686Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2410088Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2410471Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2410864Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2411263Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2411717Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2412194Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2412946Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2413691Z res = mod(**inputs) 2025-12-04T09:48:05.2414576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2415419Z outputs = self.model.decoder( 2025-12-04T09:48:05.2416303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2417147Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2417849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2418615Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2419405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2420188Z return func(*args, **kwargs) 2025-12-04T09:48:05.2420971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:48:05.2421849Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:05.2422677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2423452Z return func(*args, **kwargs) 2025-12-04T09:48:05.2424239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:48:05.2425110Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:05.2426041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:05.2427197Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:05.2427779Z 2025-12-04T09:48:05.2427953Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2428430Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2428873Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2429622Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2430300Z res = mod(**inputs) 2025-12-04T09:48:05.2435039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2435895Z outputs = self.model.decoder( 2025-12-04T09:48:05.2436726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2437581Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2438313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2439067Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2439903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2440685Z return func(*args, **kwargs) 2025-12-04T09:48:05.2441497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:48:05.2442441Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:05.2443339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:05.2444054Z return self.act(input) 2025-12-04T09:48:05.2444280Z 2025-12-04T09:48:05.2444424Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2444830Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2445253Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2445657Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2446061Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2446556Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2446974Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2447430Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2447872Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2448644Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2449290Z res = mod(**inputs) 2025-12-04T09:48:05.2450053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2450891Z outputs = self.model.decoder( 2025-12-04T09:48:05.2451695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2452514Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2453231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2453986Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2454773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2455536Z return func(*args, **kwargs) 2025-12-04T09:48:05.2456324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:48:05.2457200Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:05.2458007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2458784Z return func(*args, **kwargs) 2025-12-04T09:48:05.2459569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:48:05.2460436Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:05.2461346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:05.2462346Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:05.2462712Z 2025-12-04T09:48:05.2462867Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2463261Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2463683Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2464408Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2465063Z res = mod(**inputs) 2025-12-04T09:48:05.2465801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2466636Z outputs = self.model.decoder( 2025-12-04T09:48:05.2467772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2468619Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2469338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2470103Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2470911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2471743Z return func(*args, **kwargs) 2025-12-04T09:48:05.2472545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:48:05.2473477Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:05.2474340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:05.2475053Z return self.act(input) 2025-12-04T09:48:05.2475278Z 2025-12-04T09:48:05.2475420Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2475855Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2476240Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2476633Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2477026Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2477438Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2477831Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2478227Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2478672Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2479501Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2480150Z res = mod(**inputs) 2025-12-04T09:48:05.2480902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2481720Z outputs = self.model.decoder( 2025-12-04T09:48:05.2482514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2483341Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2484047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2484777Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2485561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2486332Z return func(*args, **kwargs) 2025-12-04T09:48:05.2487097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:48:05.2487968Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:05.2488785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2489554Z return func(*args, **kwargs) 2025-12-04T09:48:05.2490324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:48:05.2491187Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:05.2492111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:05.2493115Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:05.2493847Z 2025-12-04T09:48:05.2494049Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2494451Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2494902Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2495636Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2496312Z res = mod(**inputs) 2025-12-04T09:48:05.2497098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2497957Z outputs = self.model.decoder( 2025-12-04T09:48:05.2498795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2499680Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2500470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2501690Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2502506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2503416Z return func(*args, **kwargs) 2025-12-04T09:48:05.2504229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:48:05.2505155Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:05.2506057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:05.2506910Z return self.act(input) 2025-12-04T09:48:05.2507124Z 2025-12-04T09:48:05.2507346Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2507737Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2508142Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2508538Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2508921Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2509319Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2509719Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2510101Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2510551Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2511300Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2511979Z res = mod(**inputs) 2025-12-04T09:48:05.2512746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2513610Z outputs = self.model.decoder( 2025-12-04T09:48:05.2514437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2515274Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2516014Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2516790Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2517593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2518387Z return func(*args, **kwargs) 2025-12-04T09:48:05.2519298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:48:05.2520170Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:05.2520984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2521773Z return func(*args, **kwargs) 2025-12-04T09:48:05.2522561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:48:05.2523434Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:05.2524345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:05.2525354Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:05.2525721Z 2025-12-04T09:48:05.2525873Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2526258Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2526681Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2527395Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2528047Z res = mod(**inputs) 2025-12-04T09:48:05.2528784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2529668Z outputs = self.model.decoder( 2025-12-04T09:48:05.2530469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2531300Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2532025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2532776Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2533592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2534353Z return func(*args, **kwargs) 2025-12-04T09:48:05.2535138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:48:05.2536077Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:05.2536874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:05.2537565Z return self.act(input) 2025-12-04T09:48:05.2537780Z 2025-12-04T09:48:05.2537918Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2538310Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2538682Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2539065Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2539447Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2539829Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2540200Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2540583Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2541019Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2541727Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2542375Z res = mod(**inputs) 2025-12-04T09:48:05.2543126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2543938Z outputs = self.model.decoder( 2025-12-04T09:48:05.2544735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2545567Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2546267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2547262Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2548072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2548872Z return func(*args, **kwargs) 2025-12-04T09:48:05.2549668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:48:05.2550565Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:05.2551409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2552207Z return func(*args, **kwargs) 2025-12-04T09:48:05.2552996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:48:05.2553889Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:05.2554844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:05.2556016Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:05.2556465Z 2025-12-04T09:48:05.2556609Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2557012Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2557464Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2558253Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2558924Z res = mod(**inputs) 2025-12-04T09:48:05.2559700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2560600Z outputs = self.model.decoder( 2025-12-04T09:48:05.2561471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2562329Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2563093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2563852Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2564694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2565505Z return func(*args, **kwargs) 2025-12-04T09:48:05.2566315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:48:05.2567240Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:05.2568174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:05.2568879Z return self.act(input) 2025-12-04T09:48:05.2569080Z 2025-12-04T09:48:05.2569232Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2569609Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2569992Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2570371Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2570735Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2571129Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2571510Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2571883Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2572319Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2573038Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2573688Z res = mod(**inputs) 2025-12-04T09:48:05.2574428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2575259Z outputs = self.model.decoder( 2025-12-04T09:48:05.2576066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2576878Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2577584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2578342Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2579132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2579895Z return func(*args, **kwargs) 2025-12-04T09:48:05.2580681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 766, in forward 2025-12-04T09:48:05.2581564Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:05.2582377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2583154Z return func(*args, **kwargs) 2025-12-04T09:48:05.2583947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 443, in forward 2025-12-04T09:48:05.2584815Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:05.2585737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:05.2586869Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:05.2587435Z 2025-12-04T09:48:05.2587584Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2588084Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2588517Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2589309Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2589977Z res = mod(**inputs) 2025-12-04T09:48:05.2590764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1680, in forward 2025-12-04T09:48:05.2591622Z outputs = self.model.decoder( 2025-12-04T09:48:05.2592441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1039, in forward 2025-12-04T09:48:05.2593322Z layer_outputs = decoder_layer( 2025-12-04T09:48:05.2594037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:05.2594807Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:05.2595607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:05.2596392Z return func(*args, **kwargs) 2025-12-04T09:48:05.2597201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 798, in forward 2025-12-04T09:48:05.2598144Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:05.2599073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:05.2599763Z return self.act(input) 2025-12-04T09:48:05.2599978Z 2025-12-04T09:48:05.2600119Z cudagraph partition due to non gpu ops 2025-12-04T09:48:05.2600557Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2601816Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2602493Z res = mod(**inputs) 2025-12-04T09:48:05.2603266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1696, in forward 2025-12-04T09:48:05.2604138Z logits = self.lm_head(outputs[0]) 2025-12-04T09:48:05.2604389Z 2025-12-04T09:48:05.2604584Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:05.2605321Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:05.2605995Z res = mod(**inputs) 2025-12-04T09:48:05.2606764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/plbart/modeling_plbart.py", line 1702, in forward 2025-12-04T09:48:05.2607752Z loss = loss_fct(logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:48:05.2608175Z 2025-12-04T09:48:10.9241450Z Compilation time (from dynamo_timed): 13.860121871 2025-12-04T09:48:10.9691354Z pass 2025-12-04T09:48:10.9691988Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:10.9693646Z TIMING: _recursive_pre_grad_passes:0.02901 _recursive_joint_graph_passes:0.42514 _recursive_post_grad_passes:0.06051 async_compile.wait:1.10026 code_gen:4.89018 inductor_compile:8.35725 backend_compile:11.67407 gc:0.00109 entire_frame_compile:13.86012 total_wall_time:13.86012 2025-12-04T09:48:10.9695613Z STATS: call_* op count: 180 | FakeTensorMode.__torch_dispatch__:10061 | FakeTensor.__torch_dispatch__:1881 | ProxyTorchDispatchMode.__torch_dispatch__:1534 2025-12-04T09:48:10.9696647Z Dynamo produced 1 graphs covering 180 ops with 0 graph breaks (0 unique) 2025-12-04T09:48:13.6097529Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:48:13.6099496Z import pynvml # type: ignore[import] 2025-12-04T09:48:18.5830551Z 2025-12-04T09:48:24.8994742Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:48:24.8995183Z loading model: 0it [00:06, ?it/s] 2025-12-04T09:48:24.9027612Z cpu eval PegasusForCausalLM 2025-12-04T09:48:25.5054141Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:25.7509540Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:25.9779380Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:40.9080146Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9080955Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9081713Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9082221Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9082605Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9083006Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9083396Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9083791Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9084181Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9084570Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9084960Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9085334Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9085735Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9086123Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9086624Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9087284Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9087983Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9088693Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9089379Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9090193Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9091552Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9092743Z res = mod(**inputs) 2025-12-04T09:48:40.9094191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9095766Z outputs = self.model.decoder( 2025-12-04T09:48:40.9097293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9098853Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9100155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9102301Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9103806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9105316Z return func(*args, **kwargs) 2025-12-04T09:48:40.9106919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9108678Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9110101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9110944Z return func(*args, **kwargs) 2025-12-04T09:48:40.9111777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9112939Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9114714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9116903Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9117589Z 2025-12-04T09:48:40.9117874Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9118580Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9119007Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9119881Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9120539Z res = mod(**inputs) 2025-12-04T09:48:40.9121423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9122346Z outputs = self.model.decoder( 2025-12-04T09:48:40.9123751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9125082Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9125807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9126582Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9127943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9128970Z return func(*args, **kwargs) 2025-12-04T09:48:40.9129907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9132011Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9133022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9133874Z return self.act(input) 2025-12-04T09:48:40.9134174Z 2025-12-04T09:48:40.9134428Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9134891Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9135391Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9135960Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9136527Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9137436Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9138074Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9138882Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9148290Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9149542Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9150239Z res = mod(**inputs) 2025-12-04T09:48:40.9151031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9151908Z outputs = self.model.decoder( 2025-12-04T09:48:40.9152756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9153622Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9154337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9155109Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9155913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9156707Z return func(*args, **kwargs) 2025-12-04T09:48:40.9157537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9158451Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9159411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9160179Z return func(*args, **kwargs) 2025-12-04T09:48:40.9161068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9161953Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9162893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9163918Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9164305Z 2025-12-04T09:48:40.9164449Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9164850Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9165315Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9166036Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9166688Z res = mod(**inputs) 2025-12-04T09:48:40.9167510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9168340Z outputs = self.model.decoder( 2025-12-04T09:48:40.9169152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9169989Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9170682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9171432Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9172214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9172990Z return func(*args, **kwargs) 2025-12-04T09:48:40.9173773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9174695Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9175492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9176205Z return self.act(input) 2025-12-04T09:48:40.9176411Z 2025-12-04T09:48:40.9176551Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9176945Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9177337Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9177706Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9178096Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9178483Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9178858Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9179246Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9179683Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9180392Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9181040Z res = mod(**inputs) 2025-12-04T09:48:40.9181812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9182659Z outputs = self.model.decoder( 2025-12-04T09:48:40.9183452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9184295Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9185000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9185748Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9196825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9197641Z return func(*args, **kwargs) 2025-12-04T09:48:40.9198471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9199529Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9200356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9201762Z return func(*args, **kwargs) 2025-12-04T09:48:40.9202690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9203579Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9204593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9205630Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9206010Z 2025-12-04T09:48:40.9206171Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9206609Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9207071Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9207817Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9208480Z res = mod(**inputs) 2025-12-04T09:48:40.9209269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9210145Z outputs = self.model.decoder( 2025-12-04T09:48:40.9210981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9211839Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9212567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9213444Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9214212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9214992Z return func(*args, **kwargs) 2025-12-04T09:48:40.9215783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9216701Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9217486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9218194Z return self.act(input) 2025-12-04T09:48:40.9218394Z 2025-12-04T09:48:40.9218550Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9218930Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9219321Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9219707Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9220095Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9220470Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9220861Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9221249Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9221672Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9222395Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9223049Z res = mod(**inputs) 2025-12-04T09:48:40.9223798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9224651Z outputs = self.model.decoder( 2025-12-04T09:48:40.9225463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9226303Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9227110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9228048Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9228930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9229724Z return func(*args, **kwargs) 2025-12-04T09:48:40.9230555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9231509Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9232357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9233176Z return func(*args, **kwargs) 2025-12-04T09:48:40.9233997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9234904Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9235892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9236912Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9237308Z 2025-12-04T09:48:40.9237450Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9237855Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9238297Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9239161Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9239813Z res = mod(**inputs) 2025-12-04T09:48:40.9240584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9241414Z outputs = self.model.decoder( 2025-12-04T09:48:40.9242232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9243066Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9243765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9244507Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9245287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9246065Z return func(*args, **kwargs) 2025-12-04T09:48:40.9246845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9247760Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9248617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9249642Z return self.act(input) 2025-12-04T09:48:40.9249915Z 2025-12-04T09:48:40.9250064Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9250468Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9250866Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9251250Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9251647Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9252045Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9252424Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9252826Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9253275Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9254024Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9254687Z res = mod(**inputs) 2025-12-04T09:48:40.9255478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9256353Z outputs = self.model.decoder( 2025-12-04T09:48:40.9257174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9258093Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9258830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9259604Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9260435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9261237Z return func(*args, **kwargs) 2025-12-04T09:48:40.9262134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9263031Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9263914Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9264724Z return func(*args, **kwargs) 2025-12-04T09:48:40.9265548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9266443Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9267529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9268563Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9268942Z 2025-12-04T09:48:40.9269105Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9269493Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9269953Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9270697Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9271358Z res = mod(**inputs) 2025-12-04T09:48:40.9272150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9273025Z outputs = self.model.decoder( 2025-12-04T09:48:40.9273866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9274718Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9275453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9276232Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9277029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9277833Z return func(*args, **kwargs) 2025-12-04T09:48:40.9278762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9279679Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9280467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9281171Z return self.act(input) 2025-12-04T09:48:40.9281371Z 2025-12-04T09:48:40.9281509Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9281901Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9282287Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9282655Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9283038Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9283425Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9283796Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9284174Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9284607Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9285330Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9286009Z res = mod(**inputs) 2025-12-04T09:48:40.9286771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9287616Z outputs = self.model.decoder( 2025-12-04T09:48:40.9288417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9289291Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9289994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9290772Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9291537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9292320Z return func(*args, **kwargs) 2025-12-04T09:48:40.9293146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9294036Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9294839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9295615Z return func(*args, **kwargs) 2025-12-04T09:48:40.9296415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9297281Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9298208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9299207Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9299573Z 2025-12-04T09:48:40.9299730Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9300109Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9300558Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9301835Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9302500Z res = mod(**inputs) 2025-12-04T09:48:40.9303294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9304174Z outputs = self.model.decoder( 2025-12-04T09:48:40.9305022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9305871Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9306604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9307474Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9308269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9309077Z return func(*args, **kwargs) 2025-12-04T09:48:40.9309900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9310850Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9316418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9317156Z return self.act(input) 2025-12-04T09:48:40.9317378Z 2025-12-04T09:48:40.9317527Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9317928Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9318314Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9318707Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9319104Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9319482Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9319978Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9320371Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9320803Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9321541Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9322260Z res = mod(**inputs) 2025-12-04T09:48:40.9323049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9323997Z outputs = self.model.decoder( 2025-12-04T09:48:40.9324850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9325686Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9326423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9327175Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9327959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9328738Z return func(*args, **kwargs) 2025-12-04T09:48:40.9329520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9330409Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9331228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9331992Z return func(*args, **kwargs) 2025-12-04T09:48:40.9332782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9333662Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9334584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9335572Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9335951Z 2025-12-04T09:48:40.9336089Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9336479Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9336921Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9337627Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9338280Z res = mod(**inputs) 2025-12-04T09:48:40.9339045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9339872Z outputs = self.model.decoder( 2025-12-04T09:48:40.9340681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9341518Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9342223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9342951Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9343731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9344502Z return func(*args, **kwargs) 2025-12-04T09:48:40.9345273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9346190Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9347089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9347990Z return self.act(input) 2025-12-04T09:48:40.9348200Z 2025-12-04T09:48:40.9348342Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9348786Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9349179Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9349559Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9349949Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9350343Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9350769Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9351148Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9351595Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9352364Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9353025Z res = mod(**inputs) 2025-12-04T09:48:40.9353815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9354718Z outputs = self.model.decoder( 2025-12-04T09:48:40.9355543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9356408Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9357141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9357913Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9358703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9359607Z return func(*args, **kwargs) 2025-12-04T09:48:40.9360404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9361283Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9362094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9362870Z return func(*args, **kwargs) 2025-12-04T09:48:40.9363662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9364528Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9365456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9366462Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9366832Z 2025-12-04T09:48:40.9366987Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9367364Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9367803Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9368525Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9369168Z res = mod(**inputs) 2025-12-04T09:48:40.9369940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9370788Z outputs = self.model.decoder( 2025-12-04T09:48:40.9371602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9372429Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9373137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9373959Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9375089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9375873Z return func(*args, **kwargs) 2025-12-04T09:48:40.9376693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9377695Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9378561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9379298Z return self.act(input) 2025-12-04T09:48:40.9379525Z 2025-12-04T09:48:40.9379671Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9380134Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9380521Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9380923Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9381319Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9381727Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9382202Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9382579Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9383028Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9383802Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9384464Z res = mod(**inputs) 2025-12-04T09:48:40.9385249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9386121Z outputs = self.model.decoder( 2025-12-04T09:48:40.9387072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9387929Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9388654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9389427Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9390214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9391016Z return func(*args, **kwargs) 2025-12-04T09:48:40.9391834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9392748Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9393581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9394382Z return func(*args, **kwargs) 2025-12-04T09:48:40.9395198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9396105Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9397052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9398087Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9398469Z 2025-12-04T09:48:40.9398738Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9399115Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9399552Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9400272Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9401479Z res = mod(**inputs) 2025-12-04T09:48:40.9402269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9403151Z outputs = self.model.decoder( 2025-12-04T09:48:40.9403990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9404837Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9405561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9406335Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9407227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9408015Z return func(*args, **kwargs) 2025-12-04T09:48:40.9408831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9409823Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9410625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9411345Z return self.act(input) 2025-12-04T09:48:40.9411566Z 2025-12-04T09:48:40.9411753Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9412155Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9412538Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9413044Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9413467Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9413836Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9414217Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9414598Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9415019Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9415733Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9416382Z res = mod(**inputs) 2025-12-04T09:48:40.9417148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9417977Z outputs = self.model.decoder( 2025-12-04T09:48:40.9418780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9419613Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9420319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9421053Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9421833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9422609Z return func(*args, **kwargs) 2025-12-04T09:48:40.9423390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9424280Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9425115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9425889Z return func(*args, **kwargs) 2025-12-04T09:48:40.9426667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9427830Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9428793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9429824Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9430205Z 2025-12-04T09:48:40.9430349Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9430753Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9431216Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9431940Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9432615Z res = mod(**inputs) 2025-12-04T09:48:40.9433405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9434278Z outputs = self.model.decoder( 2025-12-04T09:48:40.9435097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9436071Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9436905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9437665Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9438517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9439329Z return func(*args, **kwargs) 2025-12-04T09:48:40.9440189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9441122Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9441947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9442708Z return self.act(input) 2025-12-04T09:48:40.9442923Z 2025-12-04T09:48:40.9443066Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9443471Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9443875Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9444277Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9444663Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9445066Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9445464Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9445851Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9446309Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9447059Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9447718Z res = mod(**inputs) 2025-12-04T09:48:40.9448625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9449466Z outputs = self.model.decoder( 2025-12-04T09:48:40.9450279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9451097Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9451808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9452557Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9453324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9454098Z return func(*args, **kwargs) 2025-12-04T09:48:40.9454894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9455780Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9456588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9457367Z return func(*args, **kwargs) 2025-12-04T09:48:40.9458164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9459034Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9459954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9460950Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9461315Z 2025-12-04T09:48:40.9461469Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9461845Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9462279Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9462997Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9463687Z res = mod(**inputs) 2025-12-04T09:48:40.9464440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9465280Z outputs = self.model.decoder( 2025-12-04T09:48:40.9466087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9467044Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9467950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9468761Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9469572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9470363Z return func(*args, **kwargs) 2025-12-04T09:48:40.9471214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9472170Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9472989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9473700Z return self.act(input) 2025-12-04T09:48:40.9473924Z 2025-12-04T09:48:40.9474065Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9474465Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9474846Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9475242Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9475635Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9476012Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9476407Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9476799Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9477251Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9477980Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9478652Z res = mod(**inputs) 2025-12-04T09:48:40.9479544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9480369Z outputs = self.model.decoder( 2025-12-04T09:48:40.9481177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9482009Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9482713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9483439Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9484218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9484994Z return func(*args, **kwargs) 2025-12-04T09:48:40.9485773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 410, in forward 2025-12-04T09:48:40.9486655Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:48:40.9487476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9488253Z return func(*args, **kwargs) 2025-12-04T09:48:40.9489031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 255, in forward 2025-12-04T09:48:40.9489910Z attn_output, attn_weights = attention_interface( 2025-12-04T09:48:40.9490845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/integrations/sdpa_attention.py", line 96, in sdpa_attention_forward 2025-12-04T09:48:40.9491858Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:48:40.9492227Z 2025-12-04T09:48:40.9492420Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9492807Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9493247Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9493948Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9494601Z res = mod(**inputs) 2025-12-04T09:48:40.9495393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1633, in forward 2025-12-04T09:48:40.9496233Z outputs = self.model.decoder( 2025-12-04T09:48:40.9497053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1120, in forward 2025-12-04T09:48:40.9497896Z layer_outputs = decoder_layer( 2025-12-04T09:48:40.9498711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:48:40.9504781Z return super().__call__(*args, **kwargs) 2025-12-04T09:48:40.9505607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:48:40.9506414Z return func(*args, **kwargs) 2025-12-04T09:48:40.9507333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 441, in forward 2025-12-04T09:48:40.9508274Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:48:40.9509094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:48:40.9509826Z return self.act(input) 2025-12-04T09:48:40.9510036Z 2025-12-04T09:48:40.9510197Z cudagraph partition due to non gpu ops 2025-12-04T09:48:40.9510639Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9511383Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9512057Z res = mod(**inputs) 2025-12-04T09:48:40.9512830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1649, in forward 2025-12-04T09:48:40.9513709Z logits = self.lm_head(outputs[0]) 2025-12-04T09:48:40.9513960Z 2025-12-04T09:48:40.9514167Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:48:40.9514911Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:48:40.9515568Z res = mod(**inputs) 2025-12-04T09:48:40.9516360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/pegasus/modeling_pegasus.py", line 1655, in forward 2025-12-04T09:48:40.9517368Z loss = loss_fct(logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:48:40.9517777Z 2025-12-04T09:48:47.4391677Z Compilation time (from dynamo_timed): 20.576642794 2025-12-04T09:48:47.4425279Z pass 2025-12-04T09:48:47.4425994Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:47.4428094Z TIMING: _recursive_pre_grad_passes:0.04885 _recursive_joint_graph_passes:0.92487 _recursive_post_grad_passes:0.08757 async_compile.wait:1.14974 code_gen:6.5055 inductor_compile:10.49657 backend_compile:17.00359 gc:0.00028 entire_frame_compile:20.57664 total_wall_time:20.57664 2025-12-04T09:48:47.4430157Z STATS: call_* op count: 333 | FakeTensorMode.__torch_dispatch__:18822 | FakeTensor.__torch_dispatch__:3465 | ProxyTorchDispatchMode.__torch_dispatch__:2851 2025-12-04T09:48:47.4431208Z Dynamo produced 1 graphs covering 333 ops with 0 graph breaks (0 unique) 2025-12-04T09:48:50.2618720Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:48:50.2620446Z import pynvml # type: ignore[import] 2025-12-04T09:48:55.2573669Z 2025-12-04T09:48:55.2750565Z loading model: 0it [00:00, ?it/s]If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.` 2025-12-04T09:48:55.2751757Z WARNING:transformers.models.roberta.modeling_roberta:If you want to use `RobertaLMHeadModel` as a standalone, add `is_decoder=True.` 2025-12-04T09:48:57.5612189Z We strongly recommend passing in an `attention_mask` since your input_ids may be padded. See https://huggingface.co/docs/transformers/troubleshooting#incorrect-output-when-padding-tokens-arent-masked. 2025-12-04T09:48:57.5615057Z You may ignore this warning if your `pad_token_id` (0) is identical to the `bos_token_id` (0), `eos_token_id` (2), or the `sep_token_id` (None), and your input is not padded. 2025-12-04T09:48:57.5617942Z WARNING:transformers.modeling_utils:We strongly recommend passing in an `attention_mask` since your input_ids may be padded. See https://huggingface.co/docs/transformers/troubleshooting#incorrect-output-when-padding-tokens-arent-masked. 2025-12-04T09:48:57.5620764Z You may ignore this warning if your `pad_token_id` (0) is identical to the `bos_token_id` (0), `eos_token_id` (2), or the `sep_token_id` (None), and your input is not padded. 2025-12-04T09:48:57.8139507Z 2025-12-04T09:48:57.8140215Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:48:57.8158273Z cpu eval RobertaForCausalLM 2025-12-04T09:48:58.7282392Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:59.1373523Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:48:59.5440306Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:15.0372775Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0373736Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0374514Z res = mod(**inputs) 2025-12-04T09:49:15.0375356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0376298Z outputs = self.roberta( 2025-12-04T09:49:15.0377186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 798, in forward 2025-12-04T09:49:15.0378147Z embedding_output = self.embeddings( 2025-12-04T09:49:15.0379130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 87, in forward 2025-12-04T09:49:15.0380346Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length) 2025-12-04T09:49:15.0381878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 1548, in create_position_ids_from_input_ids 2025-12-04T09:49:15.0387872Z mask = input_ids.ne(padding_idx).int() 2025-12-04T09:49:15.0388255Z 2025-12-04T09:49:15.0388406Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0388856Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0389325Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0389730Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0390124Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0390524Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0390920Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0391302Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0391697Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0392101Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0392556Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0392992Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0393471Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0394299Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0395212Z res = mod(**inputs) 2025-12-04T09:49:15.0395999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0396930Z outputs = self.roberta( 2025-12-04T09:49:15.0397863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 798, in forward 2025-12-04T09:49:15.0398818Z embedding_output = self.embeddings( 2025-12-04T09:49:15.0399839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 87, in forward 2025-12-04T09:49:15.0401636Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length) 2025-12-04T09:49:15.0403053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 1549, in create_position_ids_from_input_ids 2025-12-04T09:49:15.0404319Z incremental_indices = (torch.cumsum(mask, dim=1).type_as(mask) + past_key_values_length) * mask 2025-12-04T09:49:15.0404850Z 2025-12-04T09:49:15.0405048Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0405800Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0406521Z res = mod(**inputs) 2025-12-04T09:49:15.0407401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0408365Z outputs = self.roberta( 2025-12-04T09:49:15.0409255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 798, in forward 2025-12-04T09:49:15.0410251Z embedding_output = self.embeddings( 2025-12-04T09:49:15.0411111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 87, in forward 2025-12-04T09:49:15.0412280Z position_ids = create_position_ids_from_input_ids(input_ids, self.padding_idx, past_key_values_length) 2025-12-04T09:49:15.0413651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 1549, in create_position_ids_from_input_ids 2025-12-04T09:49:15.0414982Z incremental_indices = (torch.cumsum(mask, dim=1).type_as(mask) + past_key_values_length) * mask 2025-12-04T09:49:15.0415581Z 2025-12-04T09:49:15.0415748Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0416184Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0416791Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0417265Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0417673Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0418095Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0418558Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0418991Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0419782Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0420484Z res = mod(**inputs) 2025-12-04T09:49:15.0421357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0422284Z outputs = self.roberta( 2025-12-04T09:49:15.0423108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0424022Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0424929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0425859Z layer_outputs = layer_module( 2025-12-04T09:49:15.0426652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0429002Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0429815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0430652Z return func(*args, **kwargs) 2025-12-04T09:49:15.0431456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0432420Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0433244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0434089Z return func(*args, **kwargs) 2025-12-04T09:49:15.0434909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0435755Z self_outputs = self.self( 2025-12-04T09:49:15.0436548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0437333Z return func(*args, **kwargs) 2025-12-04T09:49:15.0438145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0439117Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0439501Z 2025-12-04T09:49:15.0439760Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0440154Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0440596Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0441319Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0441955Z res = mod(**inputs) 2025-12-04T09:49:15.0442709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0443533Z outputs = self.roberta( 2025-12-04T09:49:15.0444744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0445613Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0446444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0447290Z layer_outputs = layer_module( 2025-12-04T09:49:15.0447996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0448813Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0449621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0450420Z return func(*args, **kwargs) 2025-12-04T09:49:15.0451221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0452093Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0452944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0453774Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0454673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0455690Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0456633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0457542Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0458344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0459129Z return self.act(input) 2025-12-04T09:49:15.0459336Z 2025-12-04T09:49:15.0459491Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0459873Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0460263Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0460652Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0461185Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0461564Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0461940Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0462300Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0462777Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0463496Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0464129Z res = mod(**inputs) 2025-12-04T09:49:15.0464912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0465742Z outputs = self.roberta( 2025-12-04T09:49:15.0466517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0467646Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0468474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0469332Z layer_outputs = layer_module( 2025-12-04T09:49:15.0470050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0470812Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0471614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0472415Z return func(*args, **kwargs) 2025-12-04T09:49:15.0473217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0474099Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0474902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0475697Z return func(*args, **kwargs) 2025-12-04T09:49:15.0476490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0477339Z self_outputs = self.self( 2025-12-04T09:49:15.0478098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0478882Z return func(*args, **kwargs) 2025-12-04T09:49:15.0479789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0480724Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0481093Z 2025-12-04T09:49:15.0481247Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0481620Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0482053Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0482768Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0483408Z res = mod(**inputs) 2025-12-04T09:49:15.0484157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0484980Z outputs = self.roberta( 2025-12-04T09:49:15.0485746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0486553Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0487350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0488213Z layer_outputs = layer_module( 2025-12-04T09:49:15.0488908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0489635Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0490409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0491257Z return func(*args, **kwargs) 2025-12-04T09:49:15.0492085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0492938Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0493768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0494629Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0495495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0496475Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0497383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0498280Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0499035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0499739Z return self.act(input) 2025-12-04T09:49:15.0499939Z 2025-12-04T09:49:15.0510708Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0511287Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0511698Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0512115Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0512508Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0512897Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0513292Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0513670Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0514119Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0514875Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0515559Z res = mod(**inputs) 2025-12-04T09:49:15.0516344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0517210Z outputs = self.roberta( 2025-12-04T09:49:15.0518018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0518970Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0519777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0520607Z layer_outputs = layer_module( 2025-12-04T09:49:15.0521316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0522045Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0522833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0523614Z return func(*args, **kwargs) 2025-12-04T09:49:15.0524392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0525238Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0526029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0526804Z return func(*args, **kwargs) 2025-12-04T09:49:15.0527696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0528520Z self_outputs = self.self( 2025-12-04T09:49:15.0529252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0530075Z return func(*args, **kwargs) 2025-12-04T09:49:15.0530850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0531832Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0532203Z 2025-12-04T09:49:15.0532357Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0532731Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0533167Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0533966Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0534623Z res = mod(**inputs) 2025-12-04T09:49:15.0535359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0536190Z outputs = self.roberta( 2025-12-04T09:49:15.0536962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0537780Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0538584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0539410Z layer_outputs = layer_module( 2025-12-04T09:49:15.0540114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0540848Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0541631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0542410Z return func(*args, **kwargs) 2025-12-04T09:49:15.0543197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0544032Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0544853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0545676Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0546542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0547836Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0548788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0549722Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0550507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0551238Z return self.act(input) 2025-12-04T09:49:15.0551450Z 2025-12-04T09:49:15.0551610Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0551998Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0552388Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0552782Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0553174Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0553553Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0553945Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0554336Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0554774Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0555514Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0556233Z res = mod(**inputs) 2025-12-04T09:49:15.0556996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0557847Z outputs = self.roberta( 2025-12-04T09:49:15.0558681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0559633Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0560455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0561283Z layer_outputs = layer_module( 2025-12-04T09:49:15.0561986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0562755Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0563519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0564291Z return func(*args, **kwargs) 2025-12-04T09:49:15.0565077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0565910Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0566693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0567460Z return func(*args, **kwargs) 2025-12-04T09:49:15.0568246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0569126Z self_outputs = self.self( 2025-12-04T09:49:15.0575339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0576153Z return func(*args, **kwargs) 2025-12-04T09:49:15.0576954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0577920Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0578317Z 2025-12-04T09:49:15.0578460Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0578859Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0579295Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0580035Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0580709Z res = mod(**inputs) 2025-12-04T09:49:15.0581468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0582421Z outputs = self.roberta( 2025-12-04T09:49:15.0583193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0584019Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0584811Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0585635Z layer_outputs = layer_module( 2025-12-04T09:49:15.0586338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0587350Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0588146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0588945Z return func(*args, **kwargs) 2025-12-04T09:49:15.0589764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0590693Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0591555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0592409Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0593316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0594354Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0595330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0596256Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0597053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0597798Z return self.act(input) 2025-12-04T09:49:15.0598024Z 2025-12-04T09:49:15.0598172Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0598572Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0599055Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0599439Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0599822Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0600192Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0600573Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0601523Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0601979Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0602711Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0603385Z res = mod(**inputs) 2025-12-04T09:49:15.0604162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0604998Z outputs = self.roberta( 2025-12-04T09:49:15.0605795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0606652Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0607475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0608311Z layer_outputs = layer_module( 2025-12-04T09:49:15.0609037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0609801Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0610596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0611394Z return func(*args, **kwargs) 2025-12-04T09:49:15.0612211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0613081Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0613964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0614738Z return func(*args, **kwargs) 2025-12-04T09:49:15.0615527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0616356Z self_outputs = self.self( 2025-12-04T09:49:15.0617080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0617847Z return func(*args, **kwargs) 2025-12-04T09:49:15.0618633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0619556Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0620031Z 2025-12-04T09:49:15.0620175Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0620566Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0621004Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0621705Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0622404Z res = mod(**inputs) 2025-12-04T09:49:15.0623157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0623976Z outputs = self.roberta( 2025-12-04T09:49:15.0624807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0625635Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0626478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0627554Z layer_outputs = layer_module( 2025-12-04T09:49:15.0628280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0629048Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0629837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0630643Z return func(*args, **kwargs) 2025-12-04T09:49:15.0631518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0637106Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0637946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0638793Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0639701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0640719Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0641652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0642576Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0643377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0644198Z return self.act(input) 2025-12-04T09:49:15.0644400Z 2025-12-04T09:49:15.0644558Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0644935Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0645317Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0645698Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0646071Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0646458Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0646840Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0647216Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0647635Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0648358Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0649007Z res = mod(**inputs) 2025-12-04T09:49:15.0649747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0650564Z outputs = self.roberta( 2025-12-04T09:49:15.0651334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0652158Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0652952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0653835Z layer_outputs = layer_module( 2025-12-04T09:49:15.0654541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0655274Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0656058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0656874Z return func(*args, **kwargs) 2025-12-04T09:49:15.0657667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0658531Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0659323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0660128Z return func(*args, **kwargs) 2025-12-04T09:49:15.0660921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0661735Z self_outputs = self.self( 2025-12-04T09:49:15.0662471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0663244Z return func(*args, **kwargs) 2025-12-04T09:49:15.0664026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0664969Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0665350Z 2025-12-04T09:49:15.0665492Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0665881Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0666307Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0667128Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0667971Z res = mod(**inputs) 2025-12-04T09:49:15.0668734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0669586Z outputs = self.roberta( 2025-12-04T09:49:15.0670390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0671245Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0672064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0672924Z layer_outputs = layer_module( 2025-12-04T09:49:15.0673651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0674403Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0675205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0676007Z return func(*args, **kwargs) 2025-12-04T09:49:15.0676821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0677680Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0678532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0679474Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0680357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0681326Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0682245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0683184Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0683942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0684642Z return self.act(input) 2025-12-04T09:49:15.0684856Z 2025-12-04T09:49:15.0684994Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0685411Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0685783Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0686164Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0686541Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0686941Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0687319Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0687696Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0688114Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0688861Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0689511Z res = mod(**inputs) 2025-12-04T09:49:15.0690261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0691068Z outputs = self.roberta( 2025-12-04T09:49:15.0691839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0692670Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0693460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0694356Z layer_outputs = layer_module( 2025-12-04T09:49:15.0695462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0696241Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0697031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0697837Z return func(*args, **kwargs) 2025-12-04T09:49:15.0698660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0699602Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0700404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0701605Z return func(*args, **kwargs) 2025-12-04T09:49:15.0702426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0703258Z self_outputs = self.self( 2025-12-04T09:49:15.0704019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0704815Z return func(*args, **kwargs) 2025-12-04T09:49:15.0705631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0706575Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0707050Z 2025-12-04T09:49:15.0707194Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0707593Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0708026Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0708761Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0709429Z res = mod(**inputs) 2025-12-04T09:49:15.0710204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0711030Z outputs = self.roberta( 2025-12-04T09:49:15.0711821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0712767Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0713577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0714425Z layer_outputs = layer_module( 2025-12-04T09:49:15.0715190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0715956Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0716788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0717589Z return func(*args, **kwargs) 2025-12-04T09:49:15.0718400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0719415Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0720236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0721055Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0721925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0722893Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0723805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0724708Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0725480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0726169Z return self.act(input) 2025-12-04T09:49:15.0726380Z 2025-12-04T09:49:15.0726517Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0726907Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0727277Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0727657Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0728037Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0728420Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0728786Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0729164Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0729600Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0730299Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0730947Z res = mod(**inputs) 2025-12-04T09:49:15.0731698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0732503Z outputs = self.roberta( 2025-12-04T09:49:15.0733278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0734107Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0734910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0735714Z layer_outputs = layer_module( 2025-12-04T09:49:15.0736418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0737160Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0737940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0738700Z return func(*args, **kwargs) 2025-12-04T09:49:15.0739497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0740346Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0741159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0741938Z return func(*args, **kwargs) 2025-12-04T09:49:15.0742727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0743597Z self_outputs = self.self( 2025-12-04T09:49:15.0744310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0745075Z return func(*args, **kwargs) 2025-12-04T09:49:15.0745890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0746910Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0747457Z 2025-12-04T09:49:15.0747639Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0748041Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0748486Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0749208Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0749874Z res = mod(**inputs) 2025-12-04T09:49:15.0750649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0751501Z outputs = self.roberta( 2025-12-04T09:49:15.0752285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0753132Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0753959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0754797Z layer_outputs = layer_module( 2025-12-04T09:49:15.0755511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0756272Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0757251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0758043Z return func(*args, **kwargs) 2025-12-04T09:49:15.0758858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0759727Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0760577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0761431Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0762335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0763350Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0764273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0765196Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0765988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0766703Z return self.act(input) 2025-12-04T09:49:15.0766907Z 2025-12-04T09:49:15.0767047Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0767445Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0767836Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0768211Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0768713Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0769090Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0769501Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0769882Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0770314Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0771030Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0771661Z res = mod(**inputs) 2025-12-04T09:49:15.0772442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0773265Z outputs = self.roberta( 2025-12-04T09:49:15.0774050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0774868Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0775680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0776486Z layer_outputs = layer_module( 2025-12-04T09:49:15.0777156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0777875Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0778638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0779385Z return func(*args, **kwargs) 2025-12-04T09:49:15.0780154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0780983Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0781750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0782499Z return func(*args, **kwargs) 2025-12-04T09:49:15.0783268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0784082Z self_outputs = self.self( 2025-12-04T09:49:15.0784796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0785538Z return func(*args, **kwargs) 2025-12-04T09:49:15.0786312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0787506Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0787879Z 2025-12-04T09:49:15.0788019Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0788405Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0788840Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0789561Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0790210Z res = mod(**inputs) 2025-12-04T09:49:15.0790966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0791801Z outputs = self.roberta( 2025-12-04T09:49:15.0792567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0793418Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0794243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0795099Z layer_outputs = layer_module( 2025-12-04T09:49:15.0795808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0796572Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0797377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0798198Z return func(*args, **kwargs) 2025-12-04T09:49:15.0799122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0799968Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0801317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0802212Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0803189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0804208Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0805159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0806114Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0806919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0807647Z return self.act(input) 2025-12-04T09:49:15.0807855Z 2025-12-04T09:49:15.0807998Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0808479Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0808862Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0809260Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0809655Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0810036Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0810431Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0810817Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0811264Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0811988Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0812656Z res = mod(**inputs) 2025-12-04T09:49:15.0813428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0814357Z outputs = self.roberta( 2025-12-04T09:49:15.0815129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0815960Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0816756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0817561Z layer_outputs = layer_module( 2025-12-04T09:49:15.0818262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0819066Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0824884Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0825692Z return func(*args, **kwargs) 2025-12-04T09:49:15.0826507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0827462Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0828253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0829048Z return func(*args, **kwargs) 2025-12-04T09:49:15.0829859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0830706Z self_outputs = self.self( 2025-12-04T09:49:15.0831445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0832239Z return func(*args, **kwargs) 2025-12-04T09:49:15.0833042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0834077Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0834472Z 2025-12-04T09:49:15.0834616Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0835015Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0835503Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0836224Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0836888Z res = mod(**inputs) 2025-12-04T09:49:15.0837710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0838544Z outputs = self.roberta( 2025-12-04T09:49:15.0839455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0840285Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0841086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0841893Z layer_outputs = layer_module( 2025-12-04T09:49:15.0842588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0843331Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0844092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0844863Z return func(*args, **kwargs) 2025-12-04T09:49:15.0845649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0846498Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0847312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0848129Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0849003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0849984Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0850882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0851779Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0852550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0853237Z return self.act(input) 2025-12-04T09:49:15.0853450Z 2025-12-04T09:49:15.0853584Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0853970Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0854355Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0854720Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0855096Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0855475Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0855841Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0856219Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0856649Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0857350Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0857997Z res = mod(**inputs) 2025-12-04T09:49:15.0858751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0859583Z outputs = self.roberta( 2025-12-04T09:49:15.0860339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0861216Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0862016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0862827Z layer_outputs = layer_module( 2025-12-04T09:49:15.0863520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0864293Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0865099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0865860Z return func(*args, **kwargs) 2025-12-04T09:49:15.0866648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0867896Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0868722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0869511Z return func(*args, **kwargs) 2025-12-04T09:49:15.0870325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0871179Z self_outputs = self.self( 2025-12-04T09:49:15.0871927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0872722Z return func(*args, **kwargs) 2025-12-04T09:49:15.0873534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0874506Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0874882Z 2025-12-04T09:49:15.0875026Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0875432Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0875883Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0876609Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0877279Z res = mod(**inputs) 2025-12-04T09:49:15.0878059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0878910Z outputs = self.roberta( 2025-12-04T09:49:15.0879786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0880621Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0881490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0887269Z layer_outputs = layer_module( 2025-12-04T09:49:15.0887984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0888756Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0889561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0890338Z return func(*args, **kwargs) 2025-12-04T09:49:15.0891151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0892017Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0892866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0893696Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0894593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0895602Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0896588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0897508Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0898301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0899053Z return self.act(input) 2025-12-04T09:49:15.0899261Z 2025-12-04T09:49:15.0899409Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0899808Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0900234Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0900613Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0901398Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0901790Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0902258Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0902638Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0903084Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0903820Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0904477Z res = mod(**inputs) 2025-12-04T09:49:15.0905255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0906103Z outputs = self.roberta( 2025-12-04T09:49:15.0907003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0907857Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0908691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0909545Z layer_outputs = layer_module( 2025-12-04T09:49:15.0910252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0911022Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0911824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0912619Z return func(*args, **kwargs) 2025-12-04T09:49:15.0913419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 513, in forward 2025-12-04T09:49:15.0914291Z self_attention_outputs = self.attention( 2025-12-04T09:49:15.0915097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0915878Z return func(*args, **kwargs) 2025-12-04T09:49:15.0916683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 440, in forward 2025-12-04T09:49:15.0917529Z self_outputs = self.self( 2025-12-04T09:49:15.0918295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0919169Z return func(*args, **kwargs) 2025-12-04T09:49:15.0919951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 363, in forward 2025-12-04T09:49:15.0920890Z attn_output = torch.nn.functional.scaled_dot_product_attention( 2025-12-04T09:49:15.0921255Z 2025-12-04T09:49:15.0921408Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0921780Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0922215Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0922925Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0923553Z res = mod(**inputs) 2025-12-04T09:49:15.0924305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 971, in forward 2025-12-04T09:49:15.0925189Z outputs = self.roberta( 2025-12-04T09:49:15.0925958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 862, in forward 2025-12-04T09:49:15.0926767Z encoder_outputs = self.encoder( 2025-12-04T09:49:15.0927639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 606, in forward 2025-12-04T09:49:15.0928463Z layer_outputs = layer_module( 2025-12-04T09:49:15.0929186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:15.0929924Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:15.0930702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:15.0931504Z return func(*args, **kwargs) 2025-12-04T09:49:15.0932279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 543, in forward 2025-12-04T09:49:15.0933130Z layer_output = apply_chunking_to_forward( 2025-12-04T09:49:15.0933960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:49:15.0934775Z return forward_fn(*input_tensors) 2025-12-04T09:49:15.0935648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 551, in feed_forward_chunk 2025-12-04T09:49:15.0936631Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:49:15.0937538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 466, in forward 2025-12-04T09:49:15.0938423Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:49:15.0939192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:49:15.0939895Z return self.act(input) 2025-12-04T09:49:15.0940094Z 2025-12-04T09:49:15.0940245Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0940615Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0940994Z cudagraph partition due to non gpu ops 2025-12-04T09:49:15.0941425Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:15.0942116Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:15.0942755Z res = mod(**inputs) 2025-12-04T09:49:15.0943493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/roberta/modeling_roberta.py", line 994, in forward 2025-12-04T09:49:15.0944532Z lm_loss = self.loss_function( 2025-12-04T09:49:15.0945459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:49:15.0946468Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:49:15.0947612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:49:15.0948702Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:49:15.0949283Z 2025-12-04T09:49:21.8169297Z Compilation time (from dynamo_timed): 21.15034975 2025-12-04T09:49:21.8353429Z pass 2025-12-04T09:49:21.8354040Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:21.8355789Z TIMING: _recursive_pre_grad_passes:0.04672 _recursive_joint_graph_passes:0.63422 _recursive_post_grad_passes:0.09183 async_compile.wait:1.15192 code_gen:6.34707 inductor_compile:10.40405 backend_compile:16.87473 gc:0.00079 entire_frame_compile:21.15035 total_wall_time:21.15035 2025-12-04T09:49:21.8357849Z STATS: call_* op count: 303 | FakeTensorMode.__torch_dispatch__:19079 | FakeTensor.__torch_dispatch__:3569 | ProxyTorchDispatchMode.__torch_dispatch__:2812 2025-12-04T09:49:21.8359214Z Dynamo produced 1 graphs covering 303 ops with 0 graph breaks (0 unique) 2025-12-04T09:49:24.9238490Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:49:24.9240535Z import pynvml # type: ignore[import] 2025-12-04T09:49:29.8931999Z 2025-12-04T09:49:31.4180634Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:49:31.4181140Z loading model: 0it [00:01, ?it/s] 2025-12-04T09:49:31.4198110Z cpu eval T5ForConditionalGeneration 2025-12-04T09:49:32.9078313Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:33.4341741Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:33.9892272Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:51.4543107Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4544483Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4545565Z res = mod(**inputs) 2025-12-04T09:49:51.4546681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.4548246Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.4549149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4549956Z layer_outputs = layer_module( 2025-12-04T09:49:51.4550700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4551490Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4552293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4553096Z return func(*args, **kwargs) 2025-12-04T09:49:51.4553853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4554666Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4555461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4556258Z return func(*args, **kwargs) 2025-12-04T09:49:51.4557007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4557804Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4558614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4559524Z return func(*args, **kwargs) 2025-12-04T09:49:51.4560258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 549, in forward 2025-12-04T09:49:51.4561239Z position_bias = position_bias + causal_mask 2025-12-04T09:49:51.4561541Z 2025-12-04T09:49:51.4561732Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4562454Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4563117Z res = mod(**inputs) 2025-12-04T09:49:51.4563802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4564588Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4565346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4566352Z layer_outputs = layer_module( 2025-12-04T09:49:51.4567054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4567799Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4568587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4569460Z return func(*args, **kwargs) 2025-12-04T09:49:51.4570191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4571033Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4571829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4572688Z return func(*args, **kwargs) 2025-12-04T09:49:51.4581505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4582335Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4583134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4583933Z return func(*args, **kwargs) 2025-12-04T09:49:51.4584689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.4585733Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.4586008Z 2025-12-04T09:49:51.4586210Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4587175Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4588009Z res = mod(**inputs) 2025-12-04T09:49:51.4588815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4589793Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4590627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4591519Z layer_outputs = layer_module( 2025-12-04T09:49:51.4592320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4593215Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4594103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4595110Z return func(*args, **kwargs) 2025-12-04T09:49:51.4596110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4596983Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4597906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4598854Z return func(*args, **kwargs) 2025-12-04T09:49:51.4599684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4600523Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4618310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4619153Z return func(*args, **kwargs) 2025-12-04T09:49:51.4620074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.4620925Z key_states = self.k(current_states) 2025-12-04T09:49:51.4621230Z 2025-12-04T09:49:51.4621448Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4622300Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4623164Z res = mod(**inputs) 2025-12-04T09:49:51.4624020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4624842Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4625642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4626618Z layer_outputs = layer_module( 2025-12-04T09:49:51.4627688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4628606Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4629409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4630213Z return func(*args, **kwargs) 2025-12-04T09:49:51.4631020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4631822Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4632628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4633423Z return func(*args, **kwargs) 2025-12-04T09:49:51.4634176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4634968Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4635862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4638587Z return func(*args, **kwargs) 2025-12-04T09:49:51.4639326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.4640132Z value_states = self.v(current_states) 2025-12-04T09:49:51.4640418Z 2025-12-04T09:49:51.4640565Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.4640966Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.4641401Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4642149Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4642827Z res = mod(**inputs) 2025-12-04T09:49:51.4643527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4644324Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4645095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4645886Z layer_outputs = layer_module( 2025-12-04T09:49:51.4646592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4647357Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4648267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4649040Z return func(*args, **kwargs) 2025-12-04T09:49:51.4649750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4650535Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4651314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4652075Z return func(*args, **kwargs) 2025-12-04T09:49:51.4652800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4653586Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4654372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4655192Z return func(*args, **kwargs) 2025-12-04T09:49:51.4655920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.4656695Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.4656941Z 2025-12-04T09:49:51.4657183Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4657898Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4658546Z res = mod(**inputs) 2025-12-04T09:49:51.4659277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4660043Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4660852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4661620Z layer_outputs = layer_module( 2025-12-04T09:49:51.4662312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4663057Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4663837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4664617Z return func(*args, **kwargs) 2025-12-04T09:49:51.4665327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.4666134Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.4667040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.4668095Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.4668954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.4669758Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.4670024Z 2025-12-04T09:49:51.4670235Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4670961Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4671637Z res = mod(**inputs) 2025-12-04T09:49:51.4672354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4673156Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4673917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4674717Z layer_outputs = layer_module( 2025-12-04T09:49:51.4675442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4676193Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4676992Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4677785Z return func(*args, **kwargs) 2025-12-04T09:49:51.4678526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.4679437Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.4680215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.4681058Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.4681894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.4682654Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.4682930Z 2025-12-04T09:49:51.4683116Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4683877Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4684511Z res = mod(**inputs) 2025-12-04T09:49:51.4685202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4686007Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4686757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4687550Z layer_outputs = layer_module( 2025-12-04T09:49:51.4688286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4689030Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4691008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4691815Z return func(*args, **kwargs) 2025-12-04T09:49:51.4692545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.4693341Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.4694116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.4694968Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.4695806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.4696579Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.4696836Z 2025-12-04T09:49:51.4697027Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4697745Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4698486Z res = mod(**inputs) 2025-12-04T09:49:51.4699514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4700322Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4701460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4702265Z layer_outputs = layer_module( 2025-12-04T09:49:51.4702980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4703948Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4704763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4705551Z return func(*args, **kwargs) 2025-12-04T09:49:51.4706303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4707209Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4708016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4708798Z return func(*args, **kwargs) 2025-12-04T09:49:51.4709543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4710356Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4711169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4711955Z return func(*args, **kwargs) 2025-12-04T09:49:51.4712704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.4713506Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.4713767Z 2025-12-04T09:49:51.4713962Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4714818Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4715493Z res = mod(**inputs) 2025-12-04T09:49:51.4716207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4717042Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4717813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4718707Z layer_outputs = layer_module( 2025-12-04T09:49:51.4719441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4720181Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4721004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4721780Z return func(*args, **kwargs) 2025-12-04T09:49:51.4722488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4723269Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4724050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4724826Z return func(*args, **kwargs) 2025-12-04T09:49:51.4725537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4726326Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4727108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4727868Z return func(*args, **kwargs) 2025-12-04T09:49:51.4728592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.4729363Z key_states = self.k(current_states) 2025-12-04T09:49:51.4729611Z 2025-12-04T09:49:51.4729815Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4730526Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4731176Z res = mod(**inputs) 2025-12-04T09:49:51.4731866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4732637Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4733372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4734138Z layer_outputs = layer_module( 2025-12-04T09:49:51.4734834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4735592Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4736354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4737130Z return func(*args, **kwargs) 2025-12-04T09:49:51.4737850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4738614Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4739387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4740159Z return func(*args, **kwargs) 2025-12-04T09:49:51.4740875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4741644Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4742428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4743239Z return func(*args, **kwargs) 2025-12-04T09:49:51.4743949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.4744716Z value_states = self.v(current_states) 2025-12-04T09:49:51.4745024Z 2025-12-04T09:49:51.4745163Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.4745554Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.4745971Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4746819Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4747657Z res = mod(**inputs) 2025-12-04T09:49:51.4748361Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4749204Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4749988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4750784Z layer_outputs = layer_module( 2025-12-04T09:49:51.4751486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4752258Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4753064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4753855Z return func(*args, **kwargs) 2025-12-04T09:49:51.4754593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4755396Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4756208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4756984Z return func(*args, **kwargs) 2025-12-04T09:49:51.4757732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4758538Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4759342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4760227Z return func(*args, **kwargs) 2025-12-04T09:49:51.4761320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.4762137Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.4762390Z 2025-12-04T09:49:51.4762603Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4763327Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4764084Z res = mod(**inputs) 2025-12-04T09:49:51.4764803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4765653Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4766425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4767211Z layer_outputs = layer_module( 2025-12-04T09:49:51.4767929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4768674Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4769476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4770268Z return func(*args, **kwargs) 2025-12-04T09:49:51.4771003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.4771815Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.4772679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.4773654Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.4774478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.4775293Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.4775550Z 2025-12-04T09:49:51.4775751Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4776501Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4777150Z res = mod(**inputs) 2025-12-04T09:49:51.4777843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4778647Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4779381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4780143Z layer_outputs = layer_module( 2025-12-04T09:49:51.4780837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4781582Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4782343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4783126Z return func(*args, **kwargs) 2025-12-04T09:49:51.4783854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.4784636Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.4785420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.4786266Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.4787371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.4788163Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.4788452Z 2025-12-04T09:49:51.4788644Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4789388Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4790044Z res = mod(**inputs) 2025-12-04T09:49:51.4790763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4791562Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4792333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4793110Z layer_outputs = layer_module( 2025-12-04T09:49:51.4793826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4794590Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4795388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4796175Z return func(*args, **kwargs) 2025-12-04T09:49:51.4796920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.4797734Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.4798526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.4799499Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.4800332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.4801780Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.4802051Z 2025-12-04T09:49:51.4802247Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4802983Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4803732Z res = mod(**inputs) 2025-12-04T09:49:51.4804436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4805232Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4806057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4806847Z layer_outputs = layer_module( 2025-12-04T09:49:51.4807591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4808357Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4809159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4809958Z return func(*args, **kwargs) 2025-12-04T09:49:51.4810694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4811505Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4812303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4813089Z return func(*args, **kwargs) 2025-12-04T09:49:51.4813939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4814724Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4815503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4816258Z return func(*args, **kwargs) 2025-12-04T09:49:51.4816979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.4817749Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.4818003Z 2025-12-04T09:49:51.4818192Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4818911Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4819559Z res = mod(**inputs) 2025-12-04T09:49:51.4820256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4821020Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4821768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4822541Z layer_outputs = layer_module( 2025-12-04T09:49:51.4823298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4828309Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4829122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4829933Z return func(*args, **kwargs) 2025-12-04T09:49:51.4830670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4831479Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4832289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4833086Z return func(*args, **kwargs) 2025-12-04T09:49:51.4833822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4834711Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4835517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4836299Z return func(*args, **kwargs) 2025-12-04T09:49:51.4837047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.4837873Z key_states = self.k(current_states) 2025-12-04T09:49:51.4838131Z 2025-12-04T09:49:51.4838446Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4839186Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4839840Z res = mod(**inputs) 2025-12-04T09:49:51.4840539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4841335Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4842086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4842847Z layer_outputs = layer_module( 2025-12-04T09:49:51.4843541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4844269Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4845040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4845810Z return func(*args, **kwargs) 2025-12-04T09:49:51.4846534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4847293Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4848067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4848839Z return func(*args, **kwargs) 2025-12-04T09:49:51.4849548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4850332Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4851112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4851886Z return func(*args, **kwargs) 2025-12-04T09:49:51.4852589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.4853359Z value_states = self.v(current_states) 2025-12-04T09:49:51.4853616Z 2025-12-04T09:49:51.4853766Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.4854137Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.4854571Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4855287Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4855934Z res = mod(**inputs) 2025-12-04T09:49:51.4856615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4857388Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4858143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4858893Z layer_outputs = layer_module( 2025-12-04T09:49:51.4859588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4860381Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4861157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4861920Z return func(*args, **kwargs) 2025-12-04T09:49:51.4862641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4863453Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4864220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4865022Z return func(*args, **kwargs) 2025-12-04T09:49:51.4865746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4866533Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4867645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4868439Z return func(*args, **kwargs) 2025-12-04T09:49:51.4869221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.4870018Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.4870273Z 2025-12-04T09:49:51.4870465Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4871204Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4871870Z res = mod(**inputs) 2025-12-04T09:49:51.4872567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4873368Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4874137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4874926Z layer_outputs = layer_module( 2025-12-04T09:49:51.4875628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4876394Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4877186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4877968Z return func(*args, **kwargs) 2025-12-04T09:49:51.4878707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.4879618Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.4880407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.4881238Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.4882079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.4882855Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.4883110Z 2025-12-04T09:49:51.4883314Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4884018Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4884673Z res = mod(**inputs) 2025-12-04T09:49:51.4885370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4891262Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4892048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4892844Z layer_outputs = layer_module( 2025-12-04T09:49:51.4893564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4894315Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4895117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4895917Z return func(*args, **kwargs) 2025-12-04T09:49:51.4896651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.4897526Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.4898450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.4899360Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.4900186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.4901584Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.4901933Z 2025-12-04T09:49:51.4902147Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4902895Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4903552Z res = mod(**inputs) 2025-12-04T09:49:51.4904320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4905131Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4905888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4906678Z layer_outputs = layer_module( 2025-12-04T09:49:51.4907499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4908269Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4909064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4909868Z return func(*args, **kwargs) 2025-12-04T09:49:51.4910617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.4911430Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.4912247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.4913119Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.4913988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.4914777Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.4915059Z 2025-12-04T09:49:51.4915253Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4915996Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4916674Z res = mod(**inputs) 2025-12-04T09:49:51.4917372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4918173Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4919033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4919785Z layer_outputs = layer_module( 2025-12-04T09:49:51.4920482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4921216Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4921990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4922745Z return func(*args, **kwargs) 2025-12-04T09:49:51.4923466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4924240Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4925012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4925784Z return func(*args, **kwargs) 2025-12-04T09:49:51.4926570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4927355Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4928124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4928934Z return func(*args, **kwargs) 2025-12-04T09:49:51.4929656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.4930424Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.4930710Z 2025-12-04T09:49:51.4930900Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4931611Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4932259Z res = mod(**inputs) 2025-12-04T09:49:51.4932967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4933742Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4934484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4935255Z layer_outputs = layer_module( 2025-12-04T09:49:51.4935938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4936681Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4937457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4938213Z return func(*args, **kwargs) 2025-12-04T09:49:51.4938931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4939713Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4940494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4941249Z return func(*args, **kwargs) 2025-12-04T09:49:51.4941970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4942755Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4943528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4944297Z return func(*args, **kwargs) 2025-12-04T09:49:51.4945017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.4945785Z key_states = self.k(current_states) 2025-12-04T09:49:51.4946034Z 2025-12-04T09:49:51.4946223Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4947043Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4947885Z res = mod(**inputs) 2025-12-04T09:49:51.4948838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4949631Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4950401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4951191Z layer_outputs = layer_module( 2025-12-04T09:49:51.4951897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4952662Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4953521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4954321Z return func(*args, **kwargs) 2025-12-04T09:49:51.4955113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4955917Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4956722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4957533Z return func(*args, **kwargs) 2025-12-04T09:49:51.4958276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4959080Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4959916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4960802Z return func(*args, **kwargs) 2025-12-04T09:49:51.4961554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.4962332Z value_states = self.v(current_states) 2025-12-04T09:49:51.4962589Z 2025-12-04T09:49:51.4962745Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.4963122Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.4963560Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4964272Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4964917Z res = mod(**inputs) 2025-12-04T09:49:51.4965614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4966389Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4967129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4967891Z layer_outputs = layer_module( 2025-12-04T09:49:51.4968595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4969340Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4970102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4970875Z return func(*args, **kwargs) 2025-12-04T09:49:51.4971597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4972378Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4973144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4973916Z return func(*args, **kwargs) 2025-12-04T09:49:51.4974640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.4975411Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.4976202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4976986Z return func(*args, **kwargs) 2025-12-04T09:49:51.4977715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.4978469Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.4978730Z 2025-12-04T09:49:51.4978919Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4979637Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4980287Z res = mod(**inputs) 2025-12-04T09:49:51.4980982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4981762Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4982608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4983406Z layer_outputs = layer_module( 2025-12-04T09:49:51.4984109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4984856Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4985620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4986422Z return func(*args, **kwargs) 2025-12-04T09:49:51.4987408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.4988259Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.4989045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4989867Z return func(*args, **kwargs) 2025-12-04T09:49:51.4990608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:49:51.4991515Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:49:51.4991898Z 2025-12-04T09:49:51.4992090Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.4992833Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.4993500Z res = mod(**inputs) 2025-12-04T09:49:51.4994200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.4995004Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.4995777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.4996566Z layer_outputs = layer_module( 2025-12-04T09:49:51.4997276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.4998051Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.4998847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.4999719Z return func(*args, **kwargs) 2025-12-04T09:49:51.5000445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5001792Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5002615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5003476Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5004340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.5005143Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.5005412Z 2025-12-04T09:49:51.5005623Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5006346Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5007017Z res = mod(**inputs) 2025-12-04T09:49:51.5007730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5008514Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5009287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5010079Z layer_outputs = layer_module( 2025-12-04T09:49:51.5010867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5011726Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5012535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5013449Z return func(*args, **kwargs) 2025-12-04T09:49:51.5014186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5015006Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5015934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5016813Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5017715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.5018516Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.5018800Z 2025-12-04T09:49:51.5018995Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5019782Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5020436Z res = mod(**inputs) 2025-12-04T09:49:51.5021151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5021948Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5022705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5023603Z layer_outputs = layer_module( 2025-12-04T09:49:51.5024301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5025043Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5025800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5026570Z return func(*args, **kwargs) 2025-12-04T09:49:51.5027644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5028454Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5029268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5030139Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5031005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.5031794Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.5032073Z 2025-12-04T09:49:51.5032269Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5033011Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5033692Z res = mod(**inputs) 2025-12-04T09:49:51.5034400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5035199Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5035966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5036738Z layer_outputs = layer_module( 2025-12-04T09:49:51.5037455Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5038222Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5039133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5039893Z return func(*args, **kwargs) 2025-12-04T09:49:51.5040619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5041403Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5042172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5042986Z return func(*args, **kwargs) 2025-12-04T09:49:51.5043707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5044499Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5045309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5046088Z return func(*args, **kwargs) 2025-12-04T09:49:51.5046846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5047619Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5047875Z 2025-12-04T09:49:51.5048062Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5048812Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5049465Z res = mod(**inputs) 2025-12-04T09:49:51.5050149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5050929Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5051675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5052449Z layer_outputs = layer_module( 2025-12-04T09:49:51.5053135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5053886Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5054668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5055434Z return func(*args, **kwargs) 2025-12-04T09:49:51.5056164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5056947Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5057736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5058495Z return func(*args, **kwargs) 2025-12-04T09:49:51.5059229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5060006Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5060787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5061540Z return func(*args, **kwargs) 2025-12-04T09:49:51.5062267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5063029Z key_states = self.k(current_states) 2025-12-04T09:49:51.5063280Z 2025-12-04T09:49:51.5063467Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5064178Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5064828Z res = mod(**inputs) 2025-12-04T09:49:51.5065515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5066282Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5067306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5068104Z layer_outputs = layer_module( 2025-12-04T09:49:51.5068805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5069573Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5070376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5071249Z return func(*args, **kwargs) 2025-12-04T09:49:51.5071977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5072776Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5078606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5079390Z return func(*args, **kwargs) 2025-12-04T09:49:51.5080180Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5080982Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5081791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5082618Z return func(*args, **kwargs) 2025-12-04T09:49:51.5083370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5084166Z value_states = self.v(current_states) 2025-12-04T09:49:51.5084430Z 2025-12-04T09:49:51.5085063Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5085559Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5085996Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5086711Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5087345Z res = mod(**inputs) 2025-12-04T09:49:51.5088044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5088818Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5089565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5090309Z layer_outputs = layer_module( 2025-12-04T09:49:51.5091002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5091734Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5092487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5093245Z return func(*args, **kwargs) 2025-12-04T09:49:51.5093959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5094728Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5095483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5096243Z return func(*args, **kwargs) 2025-12-04T09:49:51.5096958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5097730Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5098484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5099233Z return func(*args, **kwargs) 2025-12-04T09:49:51.5099949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5100698Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5101325Z 2025-12-04T09:49:51.5101518Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5102424Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5103079Z res = mod(**inputs) 2025-12-04T09:49:51.5103781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5104569Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5105419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5106187Z layer_outputs = layer_module( 2025-12-04T09:49:51.5107020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5107831Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5108626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5109446Z return func(*args, **kwargs) 2025-12-04T09:49:51.5110189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5110997Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5111842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5112710Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5113570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.5114367Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.5114630Z 2025-12-04T09:49:51.5114819Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5115553Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5116218Z res = mod(**inputs) 2025-12-04T09:49:51.5116923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5117706Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5118461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5119341Z layer_outputs = layer_module( 2025-12-04T09:49:51.5120019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5120759Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5121524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5122296Z return func(*args, **kwargs) 2025-12-04T09:49:51.5122997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5123793Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5124566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5125391Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5126218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.5126987Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.5127243Z 2025-12-04T09:49:51.5127438Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5128127Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5128757Z res = mod(**inputs) 2025-12-04T09:49:51.5129433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5130198Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5130923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5131684Z layer_outputs = layer_module( 2025-12-04T09:49:51.5132375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5133125Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5133896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5134653Z return func(*args, **kwargs) 2025-12-04T09:49:51.5135365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5141279Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5142082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5142983Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5143822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.5144644Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.5144918Z 2025-12-04T09:49:51.5145107Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5145840Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5146484Z res = mod(**inputs) 2025-12-04T09:49:51.5147294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5148094Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5148847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5149611Z layer_outputs = layer_module( 2025-12-04T09:49:51.5150318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5151071Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5151854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5152647Z return func(*args, **kwargs) 2025-12-04T09:49:51.5153381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5154164Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5154947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5155730Z return func(*args, **kwargs) 2025-12-04T09:49:51.5156481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5157273Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5158077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5158979Z return func(*args, **kwargs) 2025-12-04T09:49:51.5159706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5160465Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5160731Z 2025-12-04T09:49:51.5160916Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5161633Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5162278Z res = mod(**inputs) 2025-12-04T09:49:51.5162959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5163729Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5164477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5165229Z layer_outputs = layer_module( 2025-12-04T09:49:51.5165928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5166753Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5179040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5179826Z return func(*args, **kwargs) 2025-12-04T09:49:51.5180530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5181387Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5182152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5182955Z return func(*args, **kwargs) 2025-12-04T09:49:51.5183664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5184425Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5185224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5185974Z return func(*args, **kwargs) 2025-12-04T09:49:51.5186677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5187834Z key_states = self.k(current_states) 2025-12-04T09:49:51.5188091Z 2025-12-04T09:49:51.5188279Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5188998Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5189645Z res = mod(**inputs) 2025-12-04T09:49:51.5190354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5191126Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5191885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5192656Z layer_outputs = layer_module( 2025-12-04T09:49:51.5193354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5194095Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5194875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5195657Z return func(*args, **kwargs) 2025-12-04T09:49:51.5196376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5197166Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5198000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5198938Z return func(*args, **kwargs) 2025-12-04T09:49:51.5199675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5200467Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5201628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5202403Z return func(*args, **kwargs) 2025-12-04T09:49:51.5203303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5204146Z value_states = self.v(current_states) 2025-12-04T09:49:51.5204405Z 2025-12-04T09:49:51.5204560Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5204942Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5205385Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5206116Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5206777Z res = mod(**inputs) 2025-12-04T09:49:51.5207593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5208384Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5209152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5209978Z layer_outputs = layer_module( 2025-12-04T09:49:51.5210697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5211453Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5212284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5213188Z return func(*args, **kwargs) 2025-12-04T09:49:51.5213944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5214715Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5215476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5216248Z return func(*args, **kwargs) 2025-12-04T09:49:51.5216963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5217737Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5218513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5219273Z return func(*args, **kwargs) 2025-12-04T09:49:51.5219993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5220750Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5221013Z 2025-12-04T09:49:51.5221202Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5221918Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5222562Z res = mod(**inputs) 2025-12-04T09:49:51.5223238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5224011Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5224753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5225507Z layer_outputs = layer_module( 2025-12-04T09:49:51.5226202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5227036Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5228003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5228786Z return func(*args, **kwargs) 2025-12-04T09:49:51.5229534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5230340Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5231131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5231926Z return func(*args, **kwargs) 2025-12-04T09:49:51.5232666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5233471Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5234254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5235039Z return func(*args, **kwargs) 2025-12-04T09:49:51.5235777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5236595Z key_states = self.k(current_states) 2025-12-04T09:49:51.5236850Z 2025-12-04T09:49:51.5237040Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5237765Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5238460Z res = mod(**inputs) 2025-12-04T09:49:51.5239258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5240023Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5240793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5241559Z layer_outputs = layer_module( 2025-12-04T09:49:51.5242265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5242996Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5243773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5244528Z return func(*args, **kwargs) 2025-12-04T09:49:51.5245243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5246022Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5246792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5247548Z return func(*args, **kwargs) 2025-12-04T09:49:51.5248264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5249040Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5249808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5250573Z return func(*args, **kwargs) 2025-12-04T09:49:51.5251294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5252058Z value_states = self.v(current_states) 2025-12-04T09:49:51.5252311Z 2025-12-04T09:49:51.5252450Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5252836Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5253266Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5253966Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5254613Z res = mod(**inputs) 2025-12-04T09:49:51.5255301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5256070Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5256803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5257570Z layer_outputs = layer_module( 2025-12-04T09:49:51.5258264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5259001Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5259763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5260604Z return func(*args, **kwargs) 2025-12-04T09:49:51.5261910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5262697Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5263501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5264295Z return func(*args, **kwargs) 2025-12-04T09:49:51.5265082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5265940Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5266853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5267687Z return func(*args, **kwargs) 2025-12-04T09:49:51.5268418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5269204Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5269464Z 2025-12-04T09:49:51.5269688Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5270425Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5271075Z res = mod(**inputs) 2025-12-04T09:49:51.5271815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5272605Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5273354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5274132Z layer_outputs = layer_module( 2025-12-04T09:49:51.5274844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5275605Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5276395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5277183Z return func(*args, **kwargs) 2025-12-04T09:49:51.5277922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5278826Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5279591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5280355Z return func(*args, **kwargs) 2025-12-04T09:49:51.5281070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5281837Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5282614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5283374Z return func(*args, **kwargs) 2025-12-04T09:49:51.5284091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5284843Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5285106Z 2025-12-04T09:49:51.5285293Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5285998Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5286632Z res = mod(**inputs) 2025-12-04T09:49:51.5287322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5288093Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5288833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5289582Z layer_outputs = layer_module( 2025-12-04T09:49:51.5290279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5291012Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5291774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5292549Z return func(*args, **kwargs) 2025-12-04T09:49:51.5293268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5294084Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5294848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5295659Z return func(*args, **kwargs) 2025-12-04T09:49:51.5296376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5297150Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5297939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5298709Z return func(*args, **kwargs) 2025-12-04T09:49:51.5299456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5300208Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5300463Z 2025-12-04T09:49:51.5300648Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5301927Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5302595Z res = mod(**inputs) 2025-12-04T09:49:51.5303292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5304095Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5304851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5305637Z layer_outputs = layer_module( 2025-12-04T09:49:51.5306348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5307181Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5307977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5308778Z return func(*args, **kwargs) 2025-12-04T09:49:51.5309520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5310307Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5311101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5311895Z return func(*args, **kwargs) 2025-12-04T09:49:51.5312621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:49:51.5313520Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:49:51.5313914Z 2025-12-04T09:49:51.5314105Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5314833Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5315485Z res = mod(**inputs) 2025-12-04T09:49:51.5316188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5316976Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5317737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5318511Z layer_outputs = layer_module( 2025-12-04T09:49:51.5319314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5320046Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5320800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5321560Z return func(*args, **kwargs) 2025-12-04T09:49:51.5322276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5323221Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5329091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5330032Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5330890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.5331679Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.5331945Z 2025-12-04T09:49:51.5332179Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5332910Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5333567Z res = mod(**inputs) 2025-12-04T09:49:51.5334305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5335099Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5335974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5336731Z layer_outputs = layer_module( 2025-12-04T09:49:51.5337411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5338138Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5338911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5339668Z return func(*args, **kwargs) 2025-12-04T09:49:51.5340392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5341186Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5341968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5342793Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5343625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.5344401Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.5344657Z 2025-12-04T09:49:51.5344854Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5345549Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5346192Z res = mod(**inputs) 2025-12-04T09:49:51.5346979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:49:51.5347926Z encoder_outputs = self.encoder( 2025-12-04T09:49:51.5348686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5349472Z layer_outputs = layer_module( 2025-12-04T09:49:51.5350185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5350929Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5351721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5352503Z return func(*args, **kwargs) 2025-12-04T09:49:51.5353229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5354029Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5354824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5355684Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5356571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.5357354Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.5357620Z 2025-12-04T09:49:51.5357819Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5358584Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5359336Z res = mod(**inputs) 2025-12-04T09:49:51.5360050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5360819Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5361549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5362352Z layer_outputs = layer_module( 2025-12-04T09:49:51.5363039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5363777Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5364530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5365292Z return func(*args, **kwargs) 2025-12-04T09:49:51.5366008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5366776Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5367550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5368310Z return func(*args, **kwargs) 2025-12-04T09:49:51.5369019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5369784Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5370558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5371324Z return func(*args, **kwargs) 2025-12-04T09:49:51.5372035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5372785Z key_states = self.k(current_states) 2025-12-04T09:49:51.5373042Z 2025-12-04T09:49:51.5373224Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5373930Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5374552Z res = mod(**inputs) 2025-12-04T09:49:51.5375230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5376003Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5376735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5377480Z layer_outputs = layer_module( 2025-12-04T09:49:51.5378168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5378893Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5379647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5380410Z return func(*args, **kwargs) 2025-12-04T09:49:51.5381122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5381895Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5382657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5383420Z return func(*args, **kwargs) 2025-12-04T09:49:51.5384203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5384969Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5385824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5391642Z return func(*args, **kwargs) 2025-12-04T09:49:51.5392382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5392513Z value_states = self.v(current_states) 2025-12-04T09:49:51.5392574Z 2025-12-04T09:49:51.5392716Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5392863Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5393052Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5393508Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5393620Z res = mod(**inputs) 2025-12-04T09:49:51.5394141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5394283Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5394801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5394931Z layer_outputs = layer_module( 2025-12-04T09:49:51.5395426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5395566Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5396107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5396225Z return func(*args, **kwargs) 2025-12-04T09:49:51.5396748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5396907Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5397433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5397553Z return func(*args, **kwargs) 2025-12-04T09:49:51.5398088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5398233Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5398772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5398890Z return func(*args, **kwargs) 2025-12-04T09:49:51.5399507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5399649Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5399666Z 2025-12-04T09:49:51.5399853Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5400264Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5400374Z res = mod(**inputs) 2025-12-04T09:49:51.5401417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5401570Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5402092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5402216Z layer_outputs = layer_module( 2025-12-04T09:49:51.5402716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5402857Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5403405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5403622Z return func(*args, **kwargs) 2025-12-04T09:49:51.5404134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5404307Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5404866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5405077Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5405650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.5405790Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.5405806Z 2025-12-04T09:49:51.5406006Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5406460Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5406575Z res = mod(**inputs) 2025-12-04T09:49:51.5407108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5407238Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5407767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5407893Z layer_outputs = layer_module( 2025-12-04T09:49:51.5408374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5408521Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5409052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5409173Z return func(*args, **kwargs) 2025-12-04T09:49:51.5409700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5409860Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5410382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5410591Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5411109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.5411255Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.5411271Z 2025-12-04T09:49:51.5411460Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5411885Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5411993Z res = mod(**inputs) 2025-12-04T09:49:51.5412515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5412651Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5413277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5413399Z layer_outputs = layer_module( 2025-12-04T09:49:51.5413872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5414009Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5414536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5414648Z return func(*args, **kwargs) 2025-12-04T09:49:51.5415147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5415312Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5415812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5416046Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5416558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.5416721Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.5416736Z 2025-12-04T09:49:51.5416934Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5417331Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5417465Z res = mod(**inputs) 2025-12-04T09:49:51.5417991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5418114Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5418659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5418782Z layer_outputs = layer_module( 2025-12-04T09:49:51.5419247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5419390Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5419908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5420022Z return func(*args, **kwargs) 2025-12-04T09:49:51.5420536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5420669Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5421196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5421315Z return func(*args, **kwargs) 2025-12-04T09:49:51.5421814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5421966Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5422477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5422593Z return func(*args, **kwargs) 2025-12-04T09:49:51.5423109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5423235Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5423250Z 2025-12-04T09:49:51.5423442Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5423844Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5423946Z res = mod(**inputs) 2025-12-04T09:49:51.5424463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5424590Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5425099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5425218Z layer_outputs = layer_module( 2025-12-04T09:49:51.5425689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5425833Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5426347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5426462Z return func(*args, **kwargs) 2025-12-04T09:49:51.5427118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5427265Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5427965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5428129Z return func(*args, **kwargs) 2025-12-04T09:49:51.5428644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5428800Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5429373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5429495Z return func(*args, **kwargs) 2025-12-04T09:49:51.5430047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5430181Z key_states = self.k(current_states) 2025-12-04T09:49:51.5430197Z 2025-12-04T09:49:51.5430396Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5430872Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5430986Z res = mod(**inputs) 2025-12-04T09:49:51.5431516Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5431643Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5432174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5432299Z layer_outputs = layer_module( 2025-12-04T09:49:51.5432780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5432935Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5433465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5433589Z return func(*args, **kwargs) 2025-12-04T09:49:51.5434115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5434256Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5434796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5434910Z return func(*args, **kwargs) 2025-12-04T09:49:51.5435428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5435579Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5436106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5436232Z return func(*args, **kwargs) 2025-12-04T09:49:51.5436744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5436874Z value_states = self.v(current_states) 2025-12-04T09:49:51.5436891Z 2025-12-04T09:49:51.5437043Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5437180Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5437369Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5437788Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5437902Z res = mod(**inputs) 2025-12-04T09:49:51.5438438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5438561Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5439081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5439219Z layer_outputs = layer_module( 2025-12-04T09:49:51.5439798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5439961Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5440488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5440599Z return func(*args, **kwargs) 2025-12-04T09:49:51.5441112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5441278Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5441786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5441946Z return func(*args, **kwargs) 2025-12-04T09:49:51.5442443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5442578Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5443141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5443258Z return func(*args, **kwargs) 2025-12-04T09:49:51.5443766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5443894Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5443912Z 2025-12-04T09:49:51.5444094Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5444508Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5444614Z res = mod(**inputs) 2025-12-04T09:49:51.5445132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5445255Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5445758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5445889Z layer_outputs = layer_module( 2025-12-04T09:49:51.5446355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5446483Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5447006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5447126Z return func(*args, **kwargs) 2025-12-04T09:49:51.5447633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5447778Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5448367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5448819Z return func(*args, **kwargs) 2025-12-04T09:49:51.5449351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5449500Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5450040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5450160Z return func(*args, **kwargs) 2025-12-04T09:49:51.5450685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5450820Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5450836Z 2025-12-04T09:49:51.5451031Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5451461Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5451572Z res = mod(**inputs) 2025-12-04T09:49:51.5452109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5452281Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5452838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5453006Z layer_outputs = layer_module( 2025-12-04T09:49:51.5453490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5453672Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5454210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5454360Z return func(*args, **kwargs) 2025-12-04T09:49:51.5454887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5455028Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5455593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5455722Z return func(*args, **kwargs) 2025-12-04T09:49:51.5456238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5456387Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5456931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5457051Z return func(*args, **kwargs) 2025-12-04T09:49:51.5457580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5457718Z key_states = self.k(current_states) 2025-12-04T09:49:51.5457734Z 2025-12-04T09:49:51.5457923Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5458351Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5458464Z res = mod(**inputs) 2025-12-04T09:49:51.5459001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5459132Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5459654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5459794Z layer_outputs = layer_module( 2025-12-04T09:49:51.5460278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5460414Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5460955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5461079Z return func(*args, **kwargs) 2025-12-04T09:49:51.5461603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5461754Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5462277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5462404Z return func(*args, **kwargs) 2025-12-04T09:49:51.5462920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5463070Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5463616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5463738Z return func(*args, **kwargs) 2025-12-04T09:49:51.5464380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5464519Z value_states = self.v(current_states) 2025-12-04T09:49:51.5464533Z 2025-12-04T09:49:51.5464702Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5464854Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5465041Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5465454Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5465563Z res = mod(**inputs) 2025-12-04T09:49:51.5466101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5466238Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5466879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5467007Z layer_outputs = layer_module( 2025-12-04T09:49:51.5467481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5467661Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5468418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5468539Z return func(*args, **kwargs) 2025-12-04T09:49:51.5469057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5469217Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5469745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5469867Z return func(*args, **kwargs) 2025-12-04T09:49:51.5470393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5470538Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5471089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5471215Z return func(*args, **kwargs) 2025-12-04T09:49:51.5471736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5471887Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5471903Z 2025-12-04T09:49:51.5472098Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5472531Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5472637Z res = mod(**inputs) 2025-12-04T09:49:51.5473160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5473307Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5473825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5473951Z layer_outputs = layer_module( 2025-12-04T09:49:51.5474453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5474585Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5475129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5475247Z return func(*args, **kwargs) 2025-12-04T09:49:51.5475758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5475928Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5476440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5476653Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5477189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.5477357Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.5477373Z 2025-12-04T09:49:51.5477576Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5477990Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5478129Z res = mod(**inputs) 2025-12-04T09:49:51.5478768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5478894Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5479436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5479561Z layer_outputs = layer_module( 2025-12-04T09:49:51.5480057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5480206Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5480724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5480840Z return func(*args, **kwargs) 2025-12-04T09:49:51.5481357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5481513Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5482027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5482237Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5482738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.5482887Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.5482902Z 2025-12-04T09:49:51.5483091Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5483503Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5483610Z res = mod(**inputs) 2025-12-04T09:49:51.5484119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5484257Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5484765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5484884Z layer_outputs = layer_module( 2025-12-04T09:49:51.5485368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5485503Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5486029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5486147Z return func(*args, **kwargs) 2025-12-04T09:49:51.5486655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5486817Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5487320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5487527Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5488037Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.5488174Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.5488189Z 2025-12-04T09:49:51.5488389Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5488792Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5488900Z res = mod(**inputs) 2025-12-04T09:49:51.5489456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5489582Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5490099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5490249Z layer_outputs = layer_module( 2025-12-04T09:49:51.5490717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5490862Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5491415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5491534Z return func(*args, **kwargs) 2025-12-04T09:49:51.5492074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5492214Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5492744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5492862Z return func(*args, **kwargs) 2025-12-04T09:49:51.5493363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5493518Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5494032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5494153Z return func(*args, **kwargs) 2025-12-04T09:49:51.5494666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5494794Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5494812Z 2025-12-04T09:49:51.5495011Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5495412Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5495519Z res = mod(**inputs) 2025-12-04T09:49:51.5496040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5496166Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5496685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5496803Z layer_outputs = layer_module( 2025-12-04T09:49:51.5497272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5497420Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5497935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5498058Z return func(*args, **kwargs) 2025-12-04T09:49:51.5498573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5498714Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5499239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5499360Z return func(*args, **kwargs) 2025-12-04T09:49:51.5499864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5500022Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5500533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5500648Z return func(*args, **kwargs) 2025-12-04T09:49:51.5501556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5501912Z key_states = self.k(current_states) 2025-12-04T09:49:51.5501938Z 2025-12-04T09:49:51.5502145Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5502563Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5502739Z res = mod(**inputs) 2025-12-04T09:49:51.5503277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5503405Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5503978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5504107Z layer_outputs = layer_module( 2025-12-04T09:49:51.5504626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5504773Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5505306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5505425Z return func(*args, **kwargs) 2025-12-04T09:49:51.5505950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5506098Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5506634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5506860Z return func(*args, **kwargs) 2025-12-04T09:49:51.5507381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5507539Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5508072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5508192Z return func(*args, **kwargs) 2025-12-04T09:49:51.5508715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5508848Z value_states = self.v(current_states) 2025-12-04T09:49:51.5508864Z 2025-12-04T09:49:51.5509021Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5509162Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5509355Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5509782Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5509897Z res = mod(**inputs) 2025-12-04T09:49:51.5510479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5510618Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5511239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5511380Z layer_outputs = layer_module( 2025-12-04T09:49:51.5511867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5512008Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5512548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5512667Z return func(*args, **kwargs) 2025-12-04T09:49:51.5513196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5513336Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5513872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5514007Z return func(*args, **kwargs) 2025-12-04T09:49:51.5514582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5514727Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5515330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5515491Z return func(*args, **kwargs) 2025-12-04T09:49:51.5516021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5516150Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5516197Z 2025-12-04T09:49:51.5516392Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5516818Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5516928Z res = mod(**inputs) 2025-12-04T09:49:51.5517500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5517633Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5518155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5518296Z layer_outputs = layer_module( 2025-12-04T09:49:51.5518781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5518922Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5519473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5519596Z return func(*args, **kwargs) 2025-12-04T09:49:51.5520130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5520279Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5520813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5520950Z return func(*args, **kwargs) 2025-12-04T09:49:51.5521467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:49:51.5521709Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:49:51.5521738Z 2025-12-04T09:49:51.5521930Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5522343Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5522469Z res = mod(**inputs) 2025-12-04T09:49:51.5523102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5523231Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5523750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5523873Z layer_outputs = layer_module( 2025-12-04T09:49:51.5524354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5524488Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5525000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5525131Z return func(*args, **kwargs) 2025-12-04T09:49:51.5525639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5525778Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5526309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5526431Z return func(*args, **kwargs) 2025-12-04T09:49:51.5526974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5527120Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5527632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5527791Z return func(*args, **kwargs) 2025-12-04T09:49:51.5528297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5528425Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5528480Z 2025-12-04T09:49:51.5528666Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5529068Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5529188Z res = mod(**inputs) 2025-12-04T09:49:51.5529731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5529864Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5530387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5530511Z layer_outputs = layer_module( 2025-12-04T09:49:51.5531000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5531140Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5531659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5531795Z return func(*args, **kwargs) 2025-12-04T09:49:51.5532302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5532445Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5532969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5533084Z return func(*args, **kwargs) 2025-12-04T09:49:51.5533595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5533739Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5534251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5534378Z return func(*args, **kwargs) 2025-12-04T09:49:51.5534880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5535010Z key_states = self.k(current_states) 2025-12-04T09:49:51.5535037Z 2025-12-04T09:49:51.5535220Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5535620Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5535743Z res = mod(**inputs) 2025-12-04T09:49:51.5536244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5536368Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5536885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5537004Z layer_outputs = layer_module( 2025-12-04T09:49:51.5537489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5537621Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5538131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5538266Z return func(*args, **kwargs) 2025-12-04T09:49:51.5538795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5538935Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5539459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5539621Z return func(*args, **kwargs) 2025-12-04T09:49:51.5540137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5540281Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5540820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5540953Z return func(*args, **kwargs) 2025-12-04T09:49:51.5541480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5541611Z value_states = self.v(current_states) 2025-12-04T09:49:51.5541640Z 2025-12-04T09:49:51.5541776Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5541908Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5542104Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5542504Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5542610Z res = mod(**inputs) 2025-12-04T09:49:51.5543126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5543316Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5543836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5543958Z layer_outputs = layer_module( 2025-12-04T09:49:51.5544427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5544583Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5545099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5545217Z return func(*args, **kwargs) 2025-12-04T09:49:51.5545734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5545875Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5546409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5546529Z return func(*args, **kwargs) 2025-12-04T09:49:51.5547299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5547476Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5548005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5548129Z return func(*args, **kwargs) 2025-12-04T09:49:51.5548660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5548790Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5548808Z 2025-12-04T09:49:51.5549016Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5549435Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5549552Z res = mod(**inputs) 2025-12-04T09:49:51.5550088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5550220Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5550750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5550912Z layer_outputs = layer_module( 2025-12-04T09:49:51.5551392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5551537Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5552069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5552218Z return func(*args, **kwargs) 2025-12-04T09:49:51.5552749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5552940Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5553467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5553708Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5554231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.5554376Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.5554391Z 2025-12-04T09:49:51.5554584Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5555007Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5555121Z res = mod(**inputs) 2025-12-04T09:49:51.5555642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5555790Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5556315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5556442Z layer_outputs = layer_module( 2025-12-04T09:49:51.5556940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5557082Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5557626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5557749Z return func(*args, **kwargs) 2025-12-04T09:49:51.5558263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5558437Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5558957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5559273Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5559786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.5559924Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.5559941Z 2025-12-04T09:49:51.5560137Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5560536Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5560644Z res = mod(**inputs) 2025-12-04T09:49:51.5561159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5561290Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5561808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5561929Z layer_outputs = layer_module( 2025-12-04T09:49:51.5562396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5562545Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5563064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5563210Z return func(*args, **kwargs) 2025-12-04T09:49:51.5563718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5563869Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5564414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5564616Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5565146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.5565299Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.5565314Z 2025-12-04T09:49:51.5565495Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5565936Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5566046Z res = mod(**inputs) 2025-12-04T09:49:51.5566549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5566689Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5567194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5567318Z layer_outputs = layer_module( 2025-12-04T09:49:51.5567804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5567937Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5568464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5568589Z return func(*args, **kwargs) 2025-12-04T09:49:51.5569088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5569246Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5569766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5569884Z return func(*args, **kwargs) 2025-12-04T09:49:51.5570399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5570538Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5571063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5571184Z return func(*args, **kwargs) 2025-12-04T09:49:51.5571684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5571821Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5571838Z 2025-12-04T09:49:51.5572025Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5572437Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5572545Z res = mod(**inputs) 2025-12-04T09:49:51.5573126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5573271Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5578921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5579055Z layer_outputs = layer_module( 2025-12-04T09:49:51.5579552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5579695Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5580245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5580938Z return func(*args, **kwargs) 2025-12-04T09:49:51.5581458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5581623Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5582194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5582313Z return func(*args, **kwargs) 2025-12-04T09:49:51.5582871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5583018Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5583560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5583712Z return func(*args, **kwargs) 2025-12-04T09:49:51.5584237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5584377Z key_states = self.k(current_states) 2025-12-04T09:49:51.5584394Z 2025-12-04T09:49:51.5584589Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5585013Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5585127Z res = mod(**inputs) 2025-12-04T09:49:51.5585653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5585795Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5586319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5586446Z layer_outputs = layer_module( 2025-12-04T09:49:51.5587084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5587232Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5587783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5587905Z return func(*args, **kwargs) 2025-12-04T09:49:51.5588427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5588586Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5589118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5589240Z return func(*args, **kwargs) 2025-12-04T09:49:51.5589771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5589919Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5590465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5590586Z return func(*args, **kwargs) 2025-12-04T09:49:51.5591099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5591251Z value_states = self.v(current_states) 2025-12-04T09:49:51.5591268Z 2025-12-04T09:49:51.5591407Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5591559Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5591746Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5592158Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5592283Z res = mod(**inputs) 2025-12-04T09:49:51.5592807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5592989Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5593531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5593652Z layer_outputs = layer_module( 2025-12-04T09:49:51.5594146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5594313Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5594842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5595004Z return func(*args, **kwargs) 2025-12-04T09:49:51.5595519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5595659Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5596233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5596353Z return func(*args, **kwargs) 2025-12-04T09:49:51.5596881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5597024Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5597549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5597686Z return func(*args, **kwargs) 2025-12-04T09:49:51.5598200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5598329Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5598360Z 2025-12-04T09:49:51.5598553Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5599066Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5599188Z res = mod(**inputs) 2025-12-04T09:49:51.5599693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5599814Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5600332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5600452Z layer_outputs = layer_module( 2025-12-04T09:49:51.5601479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5601627Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5602165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5602298Z return func(*args, **kwargs) 2025-12-04T09:49:51.5602819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5602965Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5603511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5603630Z return func(*args, **kwargs) 2025-12-04T09:49:51.5604154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5604307Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5604834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5604971Z return func(*args, **kwargs) 2025-12-04T09:49:51.5605487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5605619Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5605648Z 2025-12-04T09:49:51.5605843Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5606344Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5606463Z res = mod(**inputs) 2025-12-04T09:49:51.5606985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5607157Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5607689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5607814Z layer_outputs = layer_module( 2025-12-04T09:49:51.5608346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5608483Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5609063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5609194Z return func(*args, **kwargs) 2025-12-04T09:49:51.5609715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5609860Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5610402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5610525Z return func(*args, **kwargs) 2025-12-04T09:49:51.5611051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5611202Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5611734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5611867Z return func(*args, **kwargs) 2025-12-04T09:49:51.5612384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5612522Z key_states = self.k(current_states) 2025-12-04T09:49:51.5612549Z 2025-12-04T09:49:51.5612740Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5613154Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5613282Z res = mod(**inputs) 2025-12-04T09:49:51.5613916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5614043Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5614558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5614675Z layer_outputs = layer_module( 2025-12-04T09:49:51.5615148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5615280Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5615796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5615926Z return func(*args, **kwargs) 2025-12-04T09:49:51.5616430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5616572Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5617099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5617217Z return func(*args, **kwargs) 2025-12-04T09:49:51.5617728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5617868Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5618387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5618547Z return func(*args, **kwargs) 2025-12-04T09:49:51.5619042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5619171Z value_states = self.v(current_states) 2025-12-04T09:49:51.5619228Z 2025-12-04T09:49:51.5619365Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5619495Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5619691Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5620120Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5620226Z res = mod(**inputs) 2025-12-04T09:49:51.5620744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5620898Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5621416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5621535Z layer_outputs = layer_module( 2025-12-04T09:49:51.5622002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5622150Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5622664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5622776Z return func(*args, **kwargs) 2025-12-04T09:49:51.5623288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5623429Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5623949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5624067Z return func(*args, **kwargs) 2025-12-04T09:49:51.5624565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5624720Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5625233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5625351Z return func(*args, **kwargs) 2025-12-04T09:49:51.5625858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5625989Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5626004Z 2025-12-04T09:49:51.5626199Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5626597Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5626781Z res = mod(**inputs) 2025-12-04T09:49:51.5627482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5627614Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5628142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5628269Z layer_outputs = layer_module( 2025-12-04T09:49:51.5628750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5628890Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5629426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5629546Z return func(*args, **kwargs) 2025-12-04T09:49:51.5630070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5630213Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5630785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5630903Z return func(*args, **kwargs) 2025-12-04T09:49:51.5631416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 652, in forward 2025-12-04T09:49:51.5631703Z layer_output = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:49:51.5631720Z 2025-12-04T09:49:51.5631910Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5632380Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5632502Z res = mod(**inputs) 2025-12-04T09:49:51.5633021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5633215Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5633735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5633858Z layer_outputs = layer_module( 2025-12-04T09:49:51.5634356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5634497Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5635044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5635159Z return func(*args, **kwargs) 2025-12-04T09:49:51.5635757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5640744Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5641267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5641483Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5642011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.5642146Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.5642162Z 2025-12-04T09:49:51.5642363Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5642776Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5642885Z res = mod(**inputs) 2025-12-04T09:49:51.5643422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5643549Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5644082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5644204Z layer_outputs = layer_module( 2025-12-04T09:49:51.5644684Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5644832Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5645359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5645480Z return func(*args, **kwargs) 2025-12-04T09:49:51.5646007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5646162Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5646687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5646898Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5647415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.5647613Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.5647629Z 2025-12-04T09:49:51.5647932Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5648324Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5648468Z res = mod(**inputs) 2025-12-04T09:49:51.5648977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5649111Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5649642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5649761Z layer_outputs = layer_module( 2025-12-04T09:49:51.5650268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5650405Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5650928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5651048Z return func(*args, **kwargs) 2025-12-04T09:49:51.5651548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5651713Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5652209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5652413Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5652920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.5653053Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.5653070Z 2025-12-04T09:49:51.5653257Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5653661Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5653768Z res = mod(**inputs) 2025-12-04T09:49:51.5654275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5654403Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5654923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5655041Z layer_outputs = layer_module( 2025-12-04T09:49:51.5655509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5655651Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5656164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5656285Z return func(*args, **kwargs) 2025-12-04T09:49:51.5656791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5656930Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5657448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5657565Z return func(*args, **kwargs) 2025-12-04T09:49:51.5658064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5658214Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5658724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5658840Z return func(*args, **kwargs) 2025-12-04T09:49:51.5659352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5659506Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5659521Z 2025-12-04T09:49:51.5659718Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5660110Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5660251Z res = mod(**inputs) 2025-12-04T09:49:51.5660768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5660889Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5661432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5661548Z layer_outputs = layer_module( 2025-12-04T09:49:51.5662038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5662185Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5662693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5662805Z return func(*args, **kwargs) 2025-12-04T09:49:51.5663313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5663447Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5663968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5664083Z return func(*args, **kwargs) 2025-12-04T09:49:51.5664575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5664722Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5665237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5665349Z return func(*args, **kwargs) 2025-12-04T09:49:51.5665858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5665980Z key_states = self.k(current_states) 2025-12-04T09:49:51.5665996Z 2025-12-04T09:49:51.5666186Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5666588Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5666693Z res = mod(**inputs) 2025-12-04T09:49:51.5667321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5667443Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5668141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5668331Z layer_outputs = layer_module( 2025-12-04T09:49:51.5668815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5668963Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5669493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5669609Z return func(*args, **kwargs) 2025-12-04T09:49:51.5670135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5670279Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5670819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5670941Z return func(*args, **kwargs) 2025-12-04T09:49:51.5671459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5671653Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5672184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5672301Z return func(*args, **kwargs) 2025-12-04T09:49:51.5672830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5672994Z value_states = self.v(current_states) 2025-12-04T09:49:51.5673010Z 2025-12-04T09:49:51.5673152Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5673314Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5673504Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5673923Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5674030Z res = mod(**inputs) 2025-12-04T09:49:51.5674602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5674746Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5685094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5685320Z layer_outputs = layer_module( 2025-12-04T09:49:51.5685842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5685978Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5686529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5686645Z return func(*args, **kwargs) 2025-12-04T09:49:51.5687151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5687314Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5687831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5687962Z return func(*args, **kwargs) 2025-12-04T09:49:51.5688462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5688605Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5689131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5689245Z return func(*args, **kwargs) 2025-12-04T09:49:51.5689765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5689893Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5689910Z 2025-12-04T09:49:51.5690101Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5690513Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5690622Z res = mod(**inputs) 2025-12-04T09:49:51.5691128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5691270Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5691780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5691912Z layer_outputs = layer_module( 2025-12-04T09:49:51.5692389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5692523Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5693052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5693173Z return func(*args, **kwargs) 2025-12-04T09:49:51.5693670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5693903Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5694417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5694581Z return func(*args, **kwargs) 2025-12-04T09:49:51.5695081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5695227Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5695781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5695897Z return func(*args, **kwargs) 2025-12-04T09:49:51.5696435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5696567Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5696583Z 2025-12-04T09:49:51.5696769Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5697179Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5697287Z res = mod(**inputs) 2025-12-04T09:49:51.5697803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5697996Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5698671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5698979Z layer_outputs = layer_module( 2025-12-04T09:49:51.5699459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5699605Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5700146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5700267Z return func(*args, **kwargs) 2025-12-04T09:49:51.5701108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5701278Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5701813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5701943Z return func(*args, **kwargs) 2025-12-04T09:49:51.5702458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5702608Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5703342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5703488Z return func(*args, **kwargs) 2025-12-04T09:49:51.5704031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5704159Z key_states = self.k(current_states) 2025-12-04T09:49:51.5704175Z 2025-12-04T09:49:51.5704371Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5704802Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5704911Z res = mod(**inputs) 2025-12-04T09:49:51.5705439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5705578Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5706098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5706234Z layer_outputs = layer_module( 2025-12-04T09:49:51.5706805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5707064Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5707616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5707734Z return func(*args, **kwargs) 2025-12-04T09:49:51.5708298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5708455Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5709021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5709159Z return func(*args, **kwargs) 2025-12-04T09:49:51.5709712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5709860Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5710410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5710528Z return func(*args, **kwargs) 2025-12-04T09:49:51.5711056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5711196Z value_states = self.v(current_states) 2025-12-04T09:49:51.5711213Z 2025-12-04T09:49:51.5711351Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5711503Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5711697Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5712114Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5712240Z res = mod(**inputs) 2025-12-04T09:49:51.5712767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5712910Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5713439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5713563Z layer_outputs = layer_module( 2025-12-04T09:49:51.5714066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5714210Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5714735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5714866Z return func(*args, **kwargs) 2025-12-04T09:49:51.5715392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5715543Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5716080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5716199Z return func(*args, **kwargs) 2025-12-04T09:49:51.5716727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5716876Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5717408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5717539Z return func(*args, **kwargs) 2025-12-04T09:49:51.5718059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5718199Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5718215Z 2025-12-04T09:49:51.5718515Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5718921Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5719066Z res = mod(**inputs) 2025-12-04T09:49:51.5719575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5719716Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5720221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5720373Z layer_outputs = layer_module( 2025-12-04T09:49:51.5720851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5721014Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5721533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5721656Z return func(*args, **kwargs) 2025-12-04T09:49:51.5722188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5722356Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5722854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5723064Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5723580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.5723711Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.5723726Z 2025-12-04T09:49:51.5723929Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5724329Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5724438Z res = mod(**inputs) 2025-12-04T09:49:51.5724957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5725080Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5725588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5725720Z layer_outputs = layer_module( 2025-12-04T09:49:51.5726186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5726333Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5726844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5726960Z return func(*args, **kwargs) 2025-12-04T09:49:51.5727477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5727633Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5728134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5728359Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5728856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.5729011Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.5729026Z 2025-12-04T09:49:51.5729211Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5729612Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5729737Z res = mod(**inputs) 2025-12-04T09:49:51.5730241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5730378Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5730887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5731039Z layer_outputs = layer_module( 2025-12-04T09:49:51.5731518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5731651Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5732166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5732330Z return func(*args, **kwargs) 2025-12-04T09:49:51.5732875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5733040Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5733587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5733817Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5734336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.5734471Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.5734486Z 2025-12-04T09:49:51.5734681Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5735080Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5735190Z res = mod(**inputs) 2025-12-04T09:49:51.5735707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5735837Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5736345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5736477Z layer_outputs = layer_module( 2025-12-04T09:49:51.5736948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5737094Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5737615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5737734Z return func(*args, **kwargs) 2025-12-04T09:49:51.5738246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5738399Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5738900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 344, in forward 2025-12-04T09:49:51.5739189Z hidden_states = hidden_states + self.dropout(forwarded_states) 2025-12-04T09:49:51.5739207Z 2025-12-04T09:49:51.5739395Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5739816Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5739932Z res = mod(**inputs) 2025-12-04T09:49:51.5740439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5740572Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5741079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5741216Z layer_outputs = layer_module( 2025-12-04T09:49:51.5741687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5741825Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5742350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5742470Z return func(*args, **kwargs) 2025-12-04T09:49:51.5742976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5743155Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5743669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5743806Z return func(*args, **kwargs) 2025-12-04T09:49:51.5744340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5744485Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5745040Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5745156Z return func(*args, **kwargs) 2025-12-04T09:49:51.5745661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5745830Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5745850Z 2025-12-04T09:49:51.5746037Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5746459Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5746566Z res = mod(**inputs) 2025-12-04T09:49:51.5747183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5747332Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5747840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5748157Z layer_outputs = layer_module( 2025-12-04T09:49:51.5748645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5748787Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5749340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5749464Z return func(*args, **kwargs) 2025-12-04T09:49:51.5749986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5750147Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5750678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5750815Z return func(*args, **kwargs) 2025-12-04T09:49:51.5751338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5751484Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5752030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5752156Z return func(*args, **kwargs) 2025-12-04T09:49:51.5752692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5752825Z key_states = self.k(current_states) 2025-12-04T09:49:51.5752843Z 2025-12-04T09:49:51.5753034Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5753463Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5753581Z res = mod(**inputs) 2025-12-04T09:49:51.5754106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5754257Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5754785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5754924Z layer_outputs = layer_module( 2025-12-04T09:49:51.5755411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5755591Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5756136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5756262Z return func(*args, **kwargs) 2025-12-04T09:49:51.5756813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5756965Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5757533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5757665Z return func(*args, **kwargs) 2025-12-04T09:49:51.5758183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5758361Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5758901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5759020Z return func(*args, **kwargs) 2025-12-04T09:49:51.5759550Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5759689Z value_states = self.v(current_states) 2025-12-04T09:49:51.5759705Z 2025-12-04T09:49:51.5759845Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5759995Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5760185Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5760677Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5760808Z res = mod(**inputs) 2025-12-04T09:49:51.5761458Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5761603Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5762129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5762259Z layer_outputs = layer_module( 2025-12-04T09:49:51.5762756Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5762896Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5763435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5763567Z return func(*args, **kwargs) 2025-12-04T09:49:51.5764082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:49:51.5764233Z self_attention_outputs = self.layer[0]( 2025-12-04T09:49:51.5764763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5764885Z return func(*args, **kwargs) 2025-12-04T09:49:51.5765478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:49:51.5765621Z attention_output = self.SelfAttention( 2025-12-04T09:49:51.5766149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5766284Z return func(*args, **kwargs) 2025-12-04T09:49:51.5766800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5766946Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5766962Z 2025-12-04T09:49:51.5767153Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5767575Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5767699Z res = mod(**inputs) 2025-12-04T09:49:51.5768268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5768413Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5768931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5769084Z layer_outputs = layer_module( 2025-12-04T09:49:51.5769576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5769711Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5770270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5770400Z return func(*args, **kwargs) 2025-12-04T09:49:51.5770945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5771106Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5771633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5771754Z return func(*args, **kwargs) 2025-12-04T09:49:51.5772292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5772441Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5773078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5773213Z return func(*args, **kwargs) 2025-12-04T09:49:51.5773713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:49:51.5773856Z query_states = self.q(hidden_states) 2025-12-04T09:49:51.5773874Z 2025-12-04T09:49:51.5774061Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5774460Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5774581Z res = mod(**inputs) 2025-12-04T09:49:51.5775087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5775225Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5775731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5775851Z layer_outputs = layer_module( 2025-12-04T09:49:51.5776332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5776469Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5776987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5777117Z return func(*args, **kwargs) 2025-12-04T09:49:51.5777617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5777763Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5778278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5778401Z return func(*args, **kwargs) 2025-12-04T09:49:51.5778911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5779059Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5779573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5779702Z return func(*args, **kwargs) 2025-12-04T09:49:51.5780209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:49:51.5780378Z key_states = self.k(current_states) 2025-12-04T09:49:51.5780394Z 2025-12-04T09:49:51.5780579Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5780982Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5781145Z res = mod(**inputs) 2025-12-04T09:49:51.5781659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5781800Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5782331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5782456Z layer_outputs = layer_module( 2025-12-04T09:49:51.5782969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5783112Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5783627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5783756Z return func(*args, **kwargs) 2025-12-04T09:49:51.5784254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5784413Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5784924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5785048Z return func(*args, **kwargs) 2025-12-04T09:49:51.5785559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5785703Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5786219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5786348Z return func(*args, **kwargs) 2025-12-04T09:49:51.5786951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:49:51.5787101Z value_states = self.v(current_states) 2025-12-04T09:49:51.5787121Z 2025-12-04T09:49:51.5787257Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5787394Z cudagraph partition due to non gpu ops 2025-12-04T09:49:51.5787774Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5788196Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5788328Z res = mod(**inputs) 2025-12-04T09:49:51.5788851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5788984Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5789520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5789651Z layer_outputs = layer_module( 2025-12-04T09:49:51.5790141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5790299Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5790828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5790960Z return func(*args, **kwargs) 2025-12-04T09:49:51.5791477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:49:51.5791638Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:49:51.5792175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5792313Z return func(*args, **kwargs) 2025-12-04T09:49:51.5792874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:49:51.5793026Z attention_output = self.EncDecAttention( 2025-12-04T09:49:51.5793574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5793729Z return func(*args, **kwargs) 2025-12-04T09:49:51.5794245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:49:51.5794431Z attn_output = self.o(attn_output) 2025-12-04T09:49:51.5794454Z 2025-12-04T09:49:51.5794656Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5795087Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5795231Z res = mod(**inputs) 2025-12-04T09:49:51.5795758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5795907Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5796427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5796551Z layer_outputs = layer_module( 2025-12-04T09:49:51.5797056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5797198Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5797739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5797860Z return func(*args, **kwargs) 2025-12-04T09:49:51.5798379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5798550Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5799070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5799293Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5799808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:49:51.5799949Z hidden_states = self.wi(hidden_states) 2025-12-04T09:49:51.5799965Z 2025-12-04T09:49:51.5800166Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5800582Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5800691Z res = mod(**inputs) 2025-12-04T09:49:51.5801730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5801871Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5802415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5802540Z layer_outputs = layer_module( 2025-12-04T09:49:51.5803027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5803184Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5803718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5803862Z return func(*args, **kwargs) 2025-12-04T09:49:51.5804381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5804539Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5805066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5805362Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5805878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:49:51.5806035Z hidden_states = self.act(hidden_states) 2025-12-04T09:49:51.5806053Z 2025-12-04T09:49:51.5806247Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5806716Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5806826Z res = mod(**inputs) 2025-12-04T09:49:51.5807392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:49:51.5807537Z decoder_outputs = self.decoder( 2025-12-04T09:49:51.5808058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:49:51.5808231Z layer_outputs = layer_module( 2025-12-04T09:49:51.5808716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:49:51.5808854Z return super().__call__(*args, **kwargs) 2025-12-04T09:49:51.5809389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:49:51.5809518Z return func(*args, **kwargs) 2025-12-04T09:49:51.5810031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:49:51.5810205Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:49:51.5810726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:49:51.5810942Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:49:51.5811468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:49:51.5811610Z hidden_states = self.wo(hidden_states) 2025-12-04T09:49:51.5811626Z 2025-12-04T09:49:51.5811831Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5812249Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5812361Z res = mod(**inputs) 2025-12-04T09:49:51.5812897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1793, in forward 2025-12-04T09:49:51.5813044Z lm_logits = self.lm_head(sequence_output) 2025-12-04T09:49:51.5813060Z 2025-12-04T09:49:51.5813374Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:49:51.5813776Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:49:51.5813881Z res = mod(**inputs) 2025-12-04T09:49:51.5814399Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1800, in forward 2025-12-04T09:49:51.5814646Z loss = loss_fct(lm_logits.view(-1, lm_logits.size(-1)), labels.view(-1)) 2025-12-04T09:49:51.5814662Z 2025-12-04T09:49:58.0488757Z Compilation time (from dynamo_timed): 22.729870169 2025-12-04T09:49:58.0709924Z pass 2025-12-04T09:49:58.0710906Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:49:58.0713935Z TIMING: _recursive_pre_grad_passes:0.08039 _recursive_joint_graph_passes:0.97588 _recursive_post_grad_passes:0.08872 async_compile.wait:1.0722 code_gen:5.71405 inductor_compile:10.08795 backend_compile:18.28017 gc:0.00037 entire_frame_compile:22.72987 total_wall_time:22.72987 2025-12-04T09:49:58.0717544Z STATS: call_* op count: 810 | FakeTensorMode.__torch_dispatch__:21989 | FakeTensor.__torch_dispatch__:3647 | ProxyTorchDispatchMode.__torch_dispatch__:4640 2025-12-04T09:49:58.0719535Z Dynamo produced 1 graphs covering 810 ops with 0 graph breaks (0 unique) 2025-12-04T09:50:00.9300225Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:50:00.9302667Z import pynvml # type: ignore[import] 2025-12-04T09:50:05.8585570Z 2025-12-04T09:50:07.4056875Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:50:07.4057306Z loading model: 0it [00:01, ?it/s] 2025-12-04T09:50:07.4074287Z cpu eval T5Small 2025-12-04T09:50:08.9859151Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:50:09.5409584Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:50:10.1379954Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:50:27.5420519Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5421344Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5421988Z res = mod(**inputs) 2025-12-04T09:50:27.5422711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.5423504Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.5424272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5425039Z layer_outputs = layer_module( 2025-12-04T09:50:27.5425750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5426503Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5427660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5428450Z return func(*args, **kwargs) 2025-12-04T09:50:27.5429210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5430030Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5430834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5431642Z return func(*args, **kwargs) 2025-12-04T09:50:27.5432396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5433213Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5434008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5434807Z return func(*args, **kwargs) 2025-12-04T09:50:27.5435569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 549, in forward 2025-12-04T09:50:27.5436393Z position_bias = position_bias + causal_mask 2025-12-04T09:50:27.5436688Z 2025-12-04T09:50:27.5436886Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5437634Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5438521Z res = mod(**inputs) 2025-12-04T09:50:27.5439338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5440127Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5440889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5441667Z layer_outputs = layer_module( 2025-12-04T09:50:27.5442359Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5443342Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5444126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5444905Z return func(*args, **kwargs) 2025-12-04T09:50:27.5445620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5446470Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5447255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5448079Z return func(*args, **kwargs) 2025-12-04T09:50:27.5448806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5449597Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5450439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5451210Z return func(*args, **kwargs) 2025-12-04T09:50:27.5451938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.5452711Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.5453041Z 2025-12-04T09:50:27.5458484Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5459552Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5460495Z res = mod(**inputs) 2025-12-04T09:50:27.5461228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5462024Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5462804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5463601Z layer_outputs = layer_module( 2025-12-04T09:50:27.5464335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5465095Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5465896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5466701Z return func(*args, **kwargs) 2025-12-04T09:50:27.5467555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5468374Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5469185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5469982Z return func(*args, **kwargs) 2025-12-04T09:50:27.5470717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5471535Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5472422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5473210Z return func(*args, **kwargs) 2025-12-04T09:50:27.5474078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.5474940Z key_states = self.k(current_states) 2025-12-04T09:50:27.5475201Z 2025-12-04T09:50:27.5475416Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5476151Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5476826Z res = mod(**inputs) 2025-12-04T09:50:27.5477554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5478611Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5479355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5480134Z layer_outputs = layer_module( 2025-12-04T09:50:27.5480841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5481740Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5482586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5483560Z return func(*args, **kwargs) 2025-12-04T09:50:27.5484888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5486316Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5487842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5488658Z return func(*args, **kwargs) 2025-12-04T09:50:27.5489525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5490976Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5492403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5493883Z return func(*args, **kwargs) 2025-12-04T09:50:27.5495216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.5496617Z value_states = self.v(current_states) 2025-12-04T09:50:27.5497131Z 2025-12-04T09:50:27.5497341Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.5497907Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.5498714Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5500039Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5500701Z res = mod(**inputs) 2025-12-04T09:50:27.5502102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5502897Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5503685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5504483Z layer_outputs = layer_module( 2025-12-04T09:50:27.5505209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5505964Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5506858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5507666Z return func(*args, **kwargs) 2025-12-04T09:50:27.5508986Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5509800Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5510612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5511420Z return func(*args, **kwargs) 2025-12-04T09:50:27.5512158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5512974Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5513788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5514569Z return func(*args, **kwargs) 2025-12-04T09:50:27.5515321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.5516441Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.5516695Z 2025-12-04T09:50:27.5516904Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5517627Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5518293Z res = mod(**inputs) 2025-12-04T09:50:27.5519094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5519888Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5520698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5521491Z layer_outputs = layer_module( 2025-12-04T09:50:27.5522208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5523009Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5523821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5524618Z return func(*args, **kwargs) 2025-12-04T09:50:27.5525364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5526169Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5526979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5527850Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5528704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.5529497Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.5529778Z 2025-12-04T09:50:27.5529974Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5530812Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5531643Z res = mod(**inputs) 2025-12-04T09:50:27.5532476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5533251Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5534003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5534758Z layer_outputs = layer_module( 2025-12-04T09:50:27.5535457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5536195Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5536960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5537729Z return func(*args, **kwargs) 2025-12-04T09:50:27.5538454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5539249Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5540019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5540872Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5541706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.5542475Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.5542750Z 2025-12-04T09:50:27.5542938Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5543653Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5544307Z res = mod(**inputs) 2025-12-04T09:50:27.5545019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5545797Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5546543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5547619Z layer_outputs = layer_module( 2025-12-04T09:50:27.5548332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5549094Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5549936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5550719Z return func(*args, **kwargs) 2025-12-04T09:50:27.5551495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5552312Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5553126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5553984Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5554852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.5555652Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.5555917Z 2025-12-04T09:50:27.5556122Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5556849Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5557515Z res = mod(**inputs) 2025-12-04T09:50:27.5558229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5559128Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5559883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5560651Z layer_outputs = layer_module( 2025-12-04T09:50:27.5561346Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5562078Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5562852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5563625Z return func(*args, **kwargs) 2025-12-04T09:50:27.5564340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5565117Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5565897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5566673Z return func(*args, **kwargs) 2025-12-04T09:50:27.5567386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5568174Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5568955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5569718Z return func(*args, **kwargs) 2025-12-04T09:50:27.5570448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.5571222Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.5571476Z 2025-12-04T09:50:27.5571678Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5572379Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5573030Z res = mod(**inputs) 2025-12-04T09:50:27.5573762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5574537Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5575268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5576075Z layer_outputs = layer_module( 2025-12-04T09:50:27.5576776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5577505Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5578676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5579486Z return func(*args, **kwargs) 2025-12-04T09:50:27.5580306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5581099Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5581910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5582708Z return func(*args, **kwargs) 2025-12-04T09:50:27.5583441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5584259Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5585065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5585869Z return func(*args, **kwargs) 2025-12-04T09:50:27.5586596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.5587537Z key_states = self.k(current_states) 2025-12-04T09:50:27.5587794Z 2025-12-04T09:50:27.5588005Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5588732Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5589397Z res = mod(**inputs) 2025-12-04T09:50:27.5590107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5590908Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5591663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5592454Z layer_outputs = layer_module( 2025-12-04T09:50:27.5593173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5593936Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5594730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5595524Z return func(*args, **kwargs) 2025-12-04T09:50:27.5596272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5597058Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5597862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5598762Z return func(*args, **kwargs) 2025-12-04T09:50:27.5599479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5600248Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5601572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5602380Z return func(*args, **kwargs) 2025-12-04T09:50:27.5603124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.5604019Z value_states = self.v(current_states) 2025-12-04T09:50:27.5604302Z 2025-12-04T09:50:27.5604448Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.5604856Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.5605295Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5606086Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5606753Z res = mod(**inputs) 2025-12-04T09:50:27.5607574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5608378Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5609148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5609997Z layer_outputs = layer_module( 2025-12-04T09:50:27.5610707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5611477Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5612278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5613060Z return func(*args, **kwargs) 2025-12-04T09:50:27.5613810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5614707Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5615489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5616244Z return func(*args, **kwargs) 2025-12-04T09:50:27.5616973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5617754Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5618537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5619294Z return func(*args, **kwargs) 2025-12-04T09:50:27.5620020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.5620790Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.5621031Z 2025-12-04T09:50:27.5621218Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5621934Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5622578Z res = mod(**inputs) 2025-12-04T09:50:27.5623267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5624026Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5624777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5625552Z layer_outputs = layer_module( 2025-12-04T09:50:27.5626230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5627081Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5628052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5628851Z return func(*args, **kwargs) 2025-12-04T09:50:27.5629593Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5630419Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5631233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5632115Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5633017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.5633812Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.5634078Z 2025-12-04T09:50:27.5634288Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5635074Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5635742Z res = mod(**inputs) 2025-12-04T09:50:27.5636490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5637298Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5638055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5638890Z layer_outputs = layer_module( 2025-12-04T09:50:27.5639614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5640398Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5646124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5646932Z return func(*args, **kwargs) 2025-12-04T09:50:27.5647688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5648495Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5649315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5650190Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5651067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.5651860Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.5652145Z 2025-12-04T09:50:27.5652339Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5653181Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5653817Z res = mod(**inputs) 2025-12-04T09:50:27.5654513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5655295Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5656051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5656806Z layer_outputs = layer_module( 2025-12-04T09:50:27.5657506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5658264Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5659026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5659808Z return func(*args, **kwargs) 2025-12-04T09:50:27.5660537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5661340Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5662107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5662953Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5663790Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.5664571Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.5664827Z 2025-12-04T09:50:27.5665015Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5665730Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5666440Z res = mod(**inputs) 2025-12-04T09:50:27.5667398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5668203Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5669018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5669809Z layer_outputs = layer_module( 2025-12-04T09:50:27.5670547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5671316Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5672120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5672935Z return func(*args, **kwargs) 2025-12-04T09:50:27.5673686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5674492Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5675292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5676079Z return func(*args, **kwargs) 2025-12-04T09:50:27.5676823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5677632Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5678424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5679324Z return func(*args, **kwargs) 2025-12-04T09:50:27.5680050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.5680824Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.5681075Z 2025-12-04T09:50:27.5681262Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5681975Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5682621Z res = mod(**inputs) 2025-12-04T09:50:27.5683311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5684065Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5684816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5685589Z layer_outputs = layer_module( 2025-12-04T09:50:27.5686275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5687019Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5687798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5688570Z return func(*args, **kwargs) 2025-12-04T09:50:27.5689275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5690058Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5690840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5691597Z return func(*args, **kwargs) 2025-12-04T09:50:27.5692321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5693102Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5693886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5694680Z return func(*args, **kwargs) 2025-12-04T09:50:27.5695402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.5696169Z key_states = self.k(current_states) 2025-12-04T09:50:27.5696417Z 2025-12-04T09:50:27.5696621Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5697356Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5698002Z res = mod(**inputs) 2025-12-04T09:50:27.5698725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5699484Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5700224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5701641Z layer_outputs = layer_module( 2025-12-04T09:50:27.5702369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5703199Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5708890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5709698Z return func(*args, **kwargs) 2025-12-04T09:50:27.5710429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5711242Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5712061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5712859Z return func(*args, **kwargs) 2025-12-04T09:50:27.5713594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5714401Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5715212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5715993Z return func(*args, **kwargs) 2025-12-04T09:50:27.5716737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.5717536Z value_states = self.v(current_states) 2025-12-04T09:50:27.5717801Z 2025-12-04T09:50:27.5717956Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.5718339Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.5718901Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5719618Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5720251Z res = mod(**inputs) 2025-12-04T09:50:27.5720954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5721736Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5722488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5723242Z layer_outputs = layer_module( 2025-12-04T09:50:27.5723941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5724686Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5725465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5726235Z return func(*args, **kwargs) 2025-12-04T09:50:27.5726970Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5727757Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5728524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5729393Z return func(*args, **kwargs) 2025-12-04T09:50:27.5730123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5730909Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5731730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5732506Z return func(*args, **kwargs) 2025-12-04T09:50:27.5733302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.5734059Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.5734323Z 2025-12-04T09:50:27.5734514Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5735265Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5735923Z res = mod(**inputs) 2025-12-04T09:50:27.5736607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5737387Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5738138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5738938Z layer_outputs = layer_module( 2025-12-04T09:50:27.5739622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5740368Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5741143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5741899Z return func(*args, **kwargs) 2025-12-04T09:50:27.5742629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5743425Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5744211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5745043Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5745890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.5746671Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.5747045Z 2025-12-04T09:50:27.5747410Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5748151Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5748821Z res = mod(**inputs) 2025-12-04T09:50:27.5749543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5750330Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5751104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5751894Z layer_outputs = layer_module( 2025-12-04T09:50:27.5752599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5753365Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5754173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5769541Z return func(*args, **kwargs) 2025-12-04T09:50:27.5770478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5771342Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5772172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5773121Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5774002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.5774855Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.5775129Z 2025-12-04T09:50:27.5775345Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5776072Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5776786Z res = mod(**inputs) 2025-12-04T09:50:27.5777509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5778314Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5779112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5779915Z layer_outputs = layer_module( 2025-12-04T09:50:27.5780641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5781505Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5782294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5783077Z return func(*args, **kwargs) 2025-12-04T09:50:27.5783807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5784594Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5785387Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5786242Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5787371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.5788215Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.5788499Z 2025-12-04T09:50:27.5788694Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5789438Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5790098Z res = mod(**inputs) 2025-12-04T09:50:27.5790822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5791626Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5792405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5793184Z layer_outputs = layer_module( 2025-12-04T09:50:27.5793907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5794681Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5795470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5796273Z return func(*args, **kwargs) 2025-12-04T09:50:27.5797024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5797831Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5798630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5799526Z return func(*args, **kwargs) 2025-12-04T09:50:27.5800259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5801634Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5802562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5803363Z return func(*args, **kwargs) 2025-12-04T09:50:27.5804119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.5804957Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.5805236Z 2025-12-04T09:50:27.5805433Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5806173Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5806891Z res = mod(**inputs) 2025-12-04T09:50:27.5807603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5808413Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5809231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5810018Z layer_outputs = layer_module( 2025-12-04T09:50:27.5810745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5811518Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5812330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5813121Z return func(*args, **kwargs) 2025-12-04T09:50:27.5814004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5814788Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5815554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5816335Z return func(*args, **kwargs) 2025-12-04T09:50:27.5817064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5817852Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5818623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5819408Z return func(*args, **kwargs) 2025-12-04T09:50:27.5820139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.5820901Z key_states = self.k(current_states) 2025-12-04T09:50:27.5821168Z 2025-12-04T09:50:27.5821362Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5822084Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5822745Z res = mod(**inputs) 2025-12-04T09:50:27.5823429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5824215Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5824962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5825733Z layer_outputs = layer_module( 2025-12-04T09:50:27.5826416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5827271Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5828323Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5829208Z return func(*args, **kwargs) 2025-12-04T09:50:27.5829967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5830780Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5831586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5832420Z return func(*args, **kwargs) 2025-12-04T09:50:27.5833170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5833982Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5834813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5835617Z return func(*args, **kwargs) 2025-12-04T09:50:27.5836397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.5837203Z value_states = self.v(current_states) 2025-12-04T09:50:27.5837470Z 2025-12-04T09:50:27.5837613Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.5838050Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.5838508Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5839234Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5839908Z res = mod(**inputs) 2025-12-04T09:50:27.5840622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5841428Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5842292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5843061Z layer_outputs = layer_module( 2025-12-04T09:50:27.5843770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5844499Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5845286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5846064Z return func(*args, **kwargs) 2025-12-04T09:50:27.5846792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5847556Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5848339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5849120Z return func(*args, **kwargs) 2025-12-04T09:50:27.5849855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5850634Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5851426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5852206Z return func(*args, **kwargs) 2025-12-04T09:50:27.5852917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.5853699Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.5853959Z 2025-12-04T09:50:27.5854150Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5854872Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5855515Z res = mod(**inputs) 2025-12-04T09:50:27.5856214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5856994Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5857734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5858510Z layer_outputs = layer_module( 2025-12-04T09:50:27.5859210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5859998Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5860758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5861529Z return func(*args, **kwargs) 2025-12-04T09:50:27.5862255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5863071Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5863830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5864645Z return func(*args, **kwargs) 2025-12-04T09:50:27.5865370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:50:27.5866266Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:50:27.5866656Z 2025-12-04T09:50:27.5866943Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5867848Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5868524Z res = mod(**inputs) 2025-12-04T09:50:27.5869224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5870034Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5870810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5871593Z layer_outputs = layer_module( 2025-12-04T09:50:27.5872322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5873094Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5873903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5874687Z return func(*args, **kwargs) 2025-12-04T09:50:27.5875440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5876261Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5877075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5877936Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5878806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.5879702Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.5879960Z 2025-12-04T09:50:27.5880149Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5880874Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5881532Z res = mod(**inputs) 2025-12-04T09:50:27.5882228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5882989Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5883739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5884519Z layer_outputs = layer_module( 2025-12-04T09:50:27.5885203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5885954Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5886734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5887510Z return func(*args, **kwargs) 2025-12-04T09:50:27.5888227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5889073Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5889869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5895990Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5896918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.5897729Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.5898000Z 2025-12-04T09:50:27.5898249Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5898974Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5899651Z res = mod(**inputs) 2025-12-04T09:50:27.5901996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5902879Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5903643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5904437Z layer_outputs = layer_module( 2025-12-04T09:50:27.5905154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5905921Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5906842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5907645Z return func(*args, **kwargs) 2025-12-04T09:50:27.5908394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5909223Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5910044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5910907Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5911776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.5912579Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.5912850Z 2025-12-04T09:50:27.5913042Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5913786Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5914454Z res = mod(**inputs) 2025-12-04T09:50:27.5915175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5915959Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5916734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5917527Z layer_outputs = layer_module( 2025-12-04T09:50:27.5918251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5919098Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5919871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5920637Z return func(*args, **kwargs) 2025-12-04T09:50:27.5921332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5922097Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5922863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5923611Z return func(*args, **kwargs) 2025-12-04T09:50:27.5924307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5925167Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5925933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5926681Z return func(*args, **kwargs) 2025-12-04T09:50:27.5927439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.5928192Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.5928438Z 2025-12-04T09:50:27.5928674Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5929370Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5930006Z res = mod(**inputs) 2025-12-04T09:50:27.5930723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5931487Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5932208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5932959Z layer_outputs = layer_module( 2025-12-04T09:50:27.5933636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5934358Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5935127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5935891Z return func(*args, **kwargs) 2025-12-04T09:50:27.5936604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5937359Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5938138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5938899Z return func(*args, **kwargs) 2025-12-04T09:50:27.5939598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5940363Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5941130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5941886Z return func(*args, **kwargs) 2025-12-04T09:50:27.5942586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.5943331Z key_states = self.k(current_states) 2025-12-04T09:50:27.5943571Z 2025-12-04T09:50:27.5943761Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5944465Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5945097Z res = mod(**inputs) 2025-12-04T09:50:27.5945776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5946539Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5947532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5948311Z layer_outputs = layer_module( 2025-12-04T09:50:27.5949010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5949758Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5950530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5951311Z return func(*args, **kwargs) 2025-12-04T09:50:27.5952046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5952916Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5958622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5959410Z return func(*args, **kwargs) 2025-12-04T09:50:27.5960191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5960976Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5961804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5962590Z return func(*args, **kwargs) 2025-12-04T09:50:27.5963307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.5964125Z value_states = self.v(current_states) 2025-12-04T09:50:27.5964396Z 2025-12-04T09:50:27.5964988Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.5965377Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.5965907Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5966607Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5967243Z res = mod(**inputs) 2025-12-04T09:50:27.5967925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5968674Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5969417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5970175Z layer_outputs = layer_module( 2025-12-04T09:50:27.5970847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5971572Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5972344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5973092Z return func(*args, **kwargs) 2025-12-04T09:50:27.5973788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.5974558Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.5975330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5976070Z return func(*args, **kwargs) 2025-12-04T09:50:27.5976782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.5977555Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.5978318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5979065Z return func(*args, **kwargs) 2025-12-04T09:50:27.5979781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.5980533Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.5980771Z 2025-12-04T09:50:27.5980968Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5981662Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5982306Z res = mod(**inputs) 2025-12-04T09:50:27.5982982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5983724Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5984453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5985209Z layer_outputs = layer_module( 2025-12-04T09:50:27.5985934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5986655Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.5987755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.5988593Z return func(*args, **kwargs) 2025-12-04T09:50:27.5989312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.5990121Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.5990956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.5991823Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.5992692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.5993490Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.5993752Z 2025-12-04T09:50:27.5993949Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.5994674Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.5995319Z res = mod(**inputs) 2025-12-04T09:50:27.5996013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.5996791Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.5997541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.5998319Z layer_outputs = layer_module( 2025-12-04T09:50:27.5999025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.5999876Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6000634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6002012Z return func(*args, **kwargs) 2025-12-04T09:50:27.6002759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6003559Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6004360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6005213Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6006061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.6006838Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.6007113Z 2025-12-04T09:50:27.6007300Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6008026Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6008673Z res = mod(**inputs) 2025-12-04T09:50:27.6009366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.6010143Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.6010897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6011657Z layer_outputs = layer_module( 2025-12-04T09:50:27.6012368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6013120Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6013903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6014856Z return func(*args, **kwargs) 2025-12-04T09:50:27.6015636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6016704Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6017486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6018403Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6019256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.6020083Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.6020351Z 2025-12-04T09:50:27.6020539Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6021303Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6021960Z res = mod(**inputs) 2025-12-04T09:50:27.6022656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.6023441Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.6024194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6024973Z layer_outputs = layer_module( 2025-12-04T09:50:27.6025670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6026421Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6027305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6028086Z return func(*args, **kwargs) 2025-12-04T09:50:27.6028813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6029599Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6030390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6031162Z return func(*args, **kwargs) 2025-12-04T09:50:27.6031889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6032681Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6033470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6034242Z return func(*args, **kwargs) 2025-12-04T09:50:27.6034976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6035760Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6036020Z 2025-12-04T09:50:27.6036202Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6036929Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6037580Z res = mod(**inputs) 2025-12-04T09:50:27.6038279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.6039141Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.6039871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6040619Z layer_outputs = layer_module( 2025-12-04T09:50:27.6041293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6042019Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6042782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6043573Z return func(*args, **kwargs) 2025-12-04T09:50:27.6044272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6045041Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6045809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6046598Z return func(*args, **kwargs) 2025-12-04T09:50:27.6047294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6048092Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6048866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6049609Z return func(*args, **kwargs) 2025-12-04T09:50:27.6050352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6051114Z key_states = self.k(current_states) 2025-12-04T09:50:27.6051359Z 2025-12-04T09:50:27.6051551Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6052239Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6052870Z res = mod(**inputs) 2025-12-04T09:50:27.6053542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.6054288Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.6055019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6055770Z layer_outputs = layer_module( 2025-12-04T09:50:27.6056461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6057182Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6057947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6058694Z return func(*args, **kwargs) 2025-12-04T09:50:27.6059398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6060150Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6060911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6061660Z return func(*args, **kwargs) 2025-12-04T09:50:27.6062352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6063117Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6063912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6064690Z return func(*args, **kwargs) 2025-12-04T09:50:27.6065401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6066175Z value_states = self.v(current_states) 2025-12-04T09:50:27.6066432Z 2025-12-04T09:50:27.6066590Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6067071Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6067698Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6068447Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6069119Z res = mod(**inputs) 2025-12-04T09:50:27.6069824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6070636Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6071413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6072239Z layer_outputs = layer_module( 2025-12-04T09:50:27.6072959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6073727Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6074567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6075356Z return func(*args, **kwargs) 2025-12-04T09:50:27.6076145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6076952Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6077792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6078750Z return func(*args, **kwargs) 2025-12-04T09:50:27.6079510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6080327Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6081118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6081918Z return func(*args, **kwargs) 2025-12-04T09:50:27.6082665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6083462Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6083726Z 2025-12-04T09:50:27.6083916Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6084650Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6085314Z res = mod(**inputs) 2025-12-04T09:50:27.6086007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6086808Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6087576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6088364Z layer_outputs = layer_module( 2025-12-04T09:50:27.6089071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6089841Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6090646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6091435Z return func(*args, **kwargs) 2025-12-04T09:50:27.6092183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6092990Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6093906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6094661Z return func(*args, **kwargs) 2025-12-04T09:50:27.6095384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6096164Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6096941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6097698Z return func(*args, **kwargs) 2025-12-04T09:50:27.6098424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6099195Z key_states = self.k(current_states) 2025-12-04T09:50:27.6099444Z 2025-12-04T09:50:27.6099634Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6100349Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6101598Z res = mod(**inputs) 2025-12-04T09:50:27.6102332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6103123Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6103978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6104777Z layer_outputs = layer_module( 2025-12-04T09:50:27.6105538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6106303Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6107202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6108045Z return func(*args, **kwargs) 2025-12-04T09:50:27.6108781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6109589Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6110390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6111175Z return func(*args, **kwargs) 2025-12-04T09:50:27.6111921Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6112725Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6113534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6114314Z return func(*args, **kwargs) 2025-12-04T09:50:27.6115062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6115861Z value_states = self.v(current_states) 2025-12-04T09:50:27.6116123Z 2025-12-04T09:50:27.6116278Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6116661Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6117111Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6117849Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6118609Z res = mod(**inputs) 2025-12-04T09:50:27.6119300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6120076Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6120822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6121578Z layer_outputs = layer_module( 2025-12-04T09:50:27.6122274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6123001Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6123753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6124516Z return func(*args, **kwargs) 2025-12-04T09:50:27.6125223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6125986Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6126745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6127504Z return func(*args, **kwargs) 2025-12-04T09:50:27.6128206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6128964Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6129733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6130532Z return func(*args, **kwargs) 2025-12-04T09:50:27.6131233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6131976Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6132253Z 2025-12-04T09:50:27.6132432Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6133128Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6133800Z res = mod(**inputs) 2025-12-04T09:50:27.6134471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6135230Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6135994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6136747Z layer_outputs = layer_module( 2025-12-04T09:50:27.6137431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6138166Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6138929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6139679Z return func(*args, **kwargs) 2025-12-04T09:50:27.6140464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6146283Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6147159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6147944Z return func(*args, **kwargs) 2025-12-04T09:50:27.6148679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6149483Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6150268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6151055Z return func(*args, **kwargs) 2025-12-04T09:50:27.6151791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6152562Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6152832Z 2025-12-04T09:50:27.6153019Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6153749Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6154408Z res = mod(**inputs) 2025-12-04T09:50:27.6155110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.6155907Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.6156657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6157433Z layer_outputs = layer_module( 2025-12-04T09:50:27.6158128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6158975Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6159741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6160487Z return func(*args, **kwargs) 2025-12-04T09:50:27.6161194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6161967Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6162727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6163532Z return func(*args, **kwargs) 2025-12-04T09:50:27.6164243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6165011Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6165803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6166559Z return func(*args, **kwargs) 2025-12-04T09:50:27.6167308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6168066Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6168305Z 2025-12-04T09:50:27.6168490Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6169224Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6169853Z res = mod(**inputs) 2025-12-04T09:50:27.6170514Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.6171261Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.6171988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6172749Z layer_outputs = layer_module( 2025-12-04T09:50:27.6173425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6174151Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6174911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6175668Z return func(*args, **kwargs) 2025-12-04T09:50:27.6176375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6177138Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6177899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6178648Z return func(*args, **kwargs) 2025-12-04T09:50:27.6179356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:50:27.6180214Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:50:27.6180584Z 2025-12-04T09:50:27.6180776Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6181471Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6182099Z res = mod(**inputs) 2025-12-04T09:50:27.6182787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.6183546Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.6184281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6185042Z layer_outputs = layer_module( 2025-12-04T09:50:27.6185731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6186454Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6187497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6188279Z return func(*args, **kwargs) 2025-12-04T09:50:27.6189018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6189820Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6190623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6191518Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6192364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.6193175Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.6193445Z 2025-12-04T09:50:27.6193628Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6194345Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6195020Z res = mod(**inputs) 2025-12-04T09:50:27.6195725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.6196507Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.6197300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6198079Z layer_outputs = layer_module( 2025-12-04T09:50:27.6198784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6199627Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6200377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6201691Z return func(*args, **kwargs) 2025-12-04T09:50:27.6202495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6208239Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6209035Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6209898Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6210740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.6211513Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.6211786Z 2025-12-04T09:50:27.6211973Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6212692Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6213365Z res = mod(**inputs) 2025-12-04T09:50:27.6214060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1727, in forward 2025-12-04T09:50:27.6214845Z encoder_outputs = self.encoder( 2025-12-04T09:50:27.6215692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6216434Z layer_outputs = layer_module( 2025-12-04T09:50:27.6217121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6217851Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6218611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6219362Z return func(*args, **kwargs) 2025-12-04T09:50:27.6220077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6220865Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6221647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6222474Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6223308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.6224070Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.6224416Z 2025-12-04T09:50:27.6224598Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6225300Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6225931Z res = mod(**inputs) 2025-12-04T09:50:27.6226606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6227694Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6228451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6229277Z layer_outputs = layer_module( 2025-12-04T09:50:27.6229974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6230729Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6231555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6232347Z return func(*args, **kwargs) 2025-12-04T09:50:27.6233072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6233866Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6234662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6235445Z return func(*args, **kwargs) 2025-12-04T09:50:27.6236165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6236955Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6237745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6238519Z return func(*args, **kwargs) 2025-12-04T09:50:27.6239254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6240118Z key_states = self.k(current_states) 2025-12-04T09:50:27.6240366Z 2025-12-04T09:50:27.6240556Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6241257Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6241897Z res = mod(**inputs) 2025-12-04T09:50:27.6242576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6243328Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6244062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6244813Z layer_outputs = layer_module( 2025-12-04T09:50:27.6245501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6246229Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6246995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6247746Z return func(*args, **kwargs) 2025-12-04T09:50:27.6248459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6249224Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6250002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6250757Z return func(*args, **kwargs) 2025-12-04T09:50:27.6251459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6252232Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6253041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6253800Z return func(*args, **kwargs) 2025-12-04T09:50:27.6254509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6254679Z value_states = self.v(current_states) 2025-12-04T09:50:27.6254694Z 2025-12-04T09:50:27.6254831Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6254959Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6255156Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6255586Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6255699Z res = mod(**inputs) 2025-12-04T09:50:27.6256234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6256361Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6256871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6256989Z layer_outputs = layer_module( 2025-12-04T09:50:27.6257450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6257595Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6258106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6258234Z return func(*args, **kwargs) 2025-12-04T09:50:27.6258732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6258871Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6259383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6259503Z return func(*args, **kwargs) 2025-12-04T09:50:27.6260002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6260152Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6260667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6260790Z return func(*args, **kwargs) 2025-12-04T09:50:27.6261292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6261415Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6261430Z 2025-12-04T09:50:27.6261619Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6262018Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6262136Z res = mod(**inputs) 2025-12-04T09:50:27.6262633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6262757Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6263267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6263390Z layer_outputs = layer_module( 2025-12-04T09:50:27.6263854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6263994Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6264499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6264622Z return func(*args, **kwargs) 2025-12-04T09:50:27.6265116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6265346Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6266198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6266411Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6267077Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.6267220Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.6267236Z 2025-12-04T09:50:27.6267458Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6267880Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6267986Z res = mod(**inputs) 2025-12-04T09:50:27.6268535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6268678Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6269197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6269330Z layer_outputs = layer_module( 2025-12-04T09:50:27.6269813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6269952Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6270495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6270615Z return func(*args, **kwargs) 2025-12-04T09:50:27.6271128Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6271301Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6271814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6272038Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6272555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.6272692Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.6272710Z 2025-12-04T09:50:27.6272909Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6273320Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6273440Z res = mod(**inputs) 2025-12-04T09:50:27.6273962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6274087Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6274621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6274750Z layer_outputs = layer_module( 2025-12-04T09:50:27.6275231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6275377Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6275912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6276038Z return func(*args, **kwargs) 2025-12-04T09:50:27.6276554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6276716Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6277238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6277451Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6277977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.6278147Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.6278163Z 2025-12-04T09:50:27.6278353Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6278770Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6278911Z res = mod(**inputs) 2025-12-04T09:50:27.6279432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6279606Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6280131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6280262Z layer_outputs = layer_module( 2025-12-04T09:50:27.6280767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6280908Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6281450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6281569Z return func(*args, **kwargs) 2025-12-04T09:50:27.6282236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6282383Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6282897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6283025Z return func(*args, **kwargs) 2025-12-04T09:50:27.6283524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6283667Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6284194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6284313Z return func(*args, **kwargs) 2025-12-04T09:50:27.6284823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6284947Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6284964Z 2025-12-04T09:50:27.6285145Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6285552Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6285656Z res = mod(**inputs) 2025-12-04T09:50:27.6286161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6286297Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6286801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6286935Z layer_outputs = layer_module( 2025-12-04T09:50:27.6287400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6287533Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6288050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6288161Z return func(*args, **kwargs) 2025-12-04T09:50:27.6288659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6288813Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6289322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6289447Z return func(*args, **kwargs) 2025-12-04T09:50:27.6289947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6290119Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6290645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6290761Z return func(*args, **kwargs) 2025-12-04T09:50:27.6291298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6291420Z key_states = self.k(current_states) 2025-12-04T09:50:27.6291435Z 2025-12-04T09:50:27.6291643Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6292048Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6292154Z res = mod(**inputs) 2025-12-04T09:50:27.6292683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6292819Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6293320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6293451Z layer_outputs = layer_module( 2025-12-04T09:50:27.6293913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6294046Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6294568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6294688Z return func(*args, **kwargs) 2025-12-04T09:50:27.6295186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6295332Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6295845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6295971Z return func(*args, **kwargs) 2025-12-04T09:50:27.6296468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6296605Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6297125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6297237Z return func(*args, **kwargs) 2025-12-04T09:50:27.6297742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6297873Z value_states = self.v(current_states) 2025-12-04T09:50:27.6297888Z 2025-12-04T09:50:27.6298021Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6298161Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6298341Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6298743Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6298853Z res = mod(**inputs) 2025-12-04T09:50:27.6299360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6299494Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6299996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6300114Z layer_outputs = layer_module( 2025-12-04T09:50:27.6300587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6300719Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6301797Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6301999Z return func(*args, **kwargs) 2025-12-04T09:50:27.6302517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6302668Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6303190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6303353Z return func(*args, **kwargs) 2025-12-04T09:50:27.6303879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6304059Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6304590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6304717Z return func(*args, **kwargs) 2025-12-04T09:50:27.6305273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6305416Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6305432Z 2025-12-04T09:50:27.6305616Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6306023Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6306138Z res = mod(**inputs) 2025-12-04T09:50:27.6306656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6306904Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6307433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6307556Z layer_outputs = layer_module( 2025-12-04T09:50:27.6315567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6315765Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6316338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6316470Z return func(*args, **kwargs) 2025-12-04T09:50:27.6316996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6317155Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6317690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6317813Z return func(*args, **kwargs) 2025-12-04T09:50:27.6318349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6318607Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6319127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6319254Z return func(*args, **kwargs) 2025-12-04T09:50:27.6319754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6319894Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6319910Z 2025-12-04T09:50:27.6320099Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6320498Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6320617Z res = mod(**inputs) 2025-12-04T09:50:27.6321125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6321251Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6321766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6321925Z layer_outputs = layer_module( 2025-12-04T09:50:27.6322487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6322624Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6323134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6323302Z return func(*args, **kwargs) 2025-12-04T09:50:27.6323802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6323952Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6324497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6324615Z return func(*args, **kwargs) 2025-12-04T09:50:27.6325156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6325303Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6325812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6325937Z return func(*args, **kwargs) 2025-12-04T09:50:27.6326434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6326574Z key_states = self.k(current_states) 2025-12-04T09:50:27.6326590Z 2025-12-04T09:50:27.6326775Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6327181Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6327299Z res = mod(**inputs) 2025-12-04T09:50:27.6327856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6328011Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6328830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6328956Z layer_outputs = layer_module( 2025-12-04T09:50:27.6329452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6329591Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6330120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6330248Z return func(*args, **kwargs) 2025-12-04T09:50:27.6330764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6330917Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6331446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6331564Z return func(*args, **kwargs) 2025-12-04T09:50:27.6332085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6332231Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6332760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6332892Z return func(*args, **kwargs) 2025-12-04T09:50:27.6333404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6333551Z value_states = self.v(current_states) 2025-12-04T09:50:27.6333568Z 2025-12-04T09:50:27.6333708Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6333844Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6334047Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6334457Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6334609Z res = mod(**inputs) 2025-12-04T09:50:27.6335137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6335263Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6335843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6335967Z layer_outputs = layer_module( 2025-12-04T09:50:27.6336478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6336628Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6337154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6337303Z return func(*args, **kwargs) 2025-12-04T09:50:27.6337831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6337973Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6338509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6338629Z return func(*args, **kwargs) 2025-12-04T09:50:27.6339140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6339293Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6339819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6339944Z return func(*args, **kwargs) 2025-12-04T09:50:27.6340462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6340591Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6340607Z 2025-12-04T09:50:27.6340804Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6341214Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6341321Z res = mod(**inputs) 2025-12-04T09:50:27.6341858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6341985Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6342522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6342647Z layer_outputs = layer_module( 2025-12-04T09:50:27.6343236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6343380Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6343889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6344013Z return func(*args, **kwargs) 2025-12-04T09:50:27.6344510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6344662Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6345170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6345375Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6345871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.6346014Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.6346029Z 2025-12-04T09:50:27.6346211Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6346619Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6346837Z res = mod(**inputs) 2025-12-04T09:50:27.6347525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6347666Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6348220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6348343Z layer_outputs = layer_module( 2025-12-04T09:50:27.6348866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6349003Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6349536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6349687Z return func(*args, **kwargs) 2025-12-04T09:50:27.6350205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6350371Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6350885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6351105Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6351621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.6351761Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.6351777Z 2025-12-04T09:50:27.6351972Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6352382Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6352494Z res = mod(**inputs) 2025-12-04T09:50:27.6353024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6353155Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6353681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6353805Z layer_outputs = layer_module( 2025-12-04T09:50:27.6354287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6354431Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6354960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6355087Z return func(*args, **kwargs) 2025-12-04T09:50:27.6355600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6355757Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6356280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6356489Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6357004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.6357151Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.6357167Z 2025-12-04T09:50:27.6357353Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6357777Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6357886Z res = mod(**inputs) 2025-12-04T09:50:27.6358407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6358550Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6359176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6359329Z layer_outputs = layer_module( 2025-12-04T09:50:27.6359801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6359934Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6360483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6360597Z return func(*args, **kwargs) 2025-12-04T09:50:27.6361129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6361279Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6361818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6361942Z return func(*args, **kwargs) 2025-12-04T09:50:27.6362442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6362578Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6363102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6363220Z return func(*args, **kwargs) 2025-12-04T09:50:27.6363716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6363856Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6363871Z 2025-12-04T09:50:27.6364049Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6364454Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6364561Z res = mod(**inputs) 2025-12-04T09:50:27.6365063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6365200Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6365702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6365818Z layer_outputs = layer_module( 2025-12-04T09:50:27.6366295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6366424Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6366951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6367066Z return func(*args, **kwargs) 2025-12-04T09:50:27.6367567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6367717Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6368234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6368353Z return func(*args, **kwargs) 2025-12-04T09:50:27.6368852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6368994Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6369512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6369627Z return func(*args, **kwargs) 2025-12-04T09:50:27.6370130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6370263Z key_states = self.k(current_states) 2025-12-04T09:50:27.6370278Z 2025-12-04T09:50:27.6370464Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6370904Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6371009Z res = mod(**inputs) 2025-12-04T09:50:27.6371513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6371639Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6372171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6372291Z layer_outputs = layer_module( 2025-12-04T09:50:27.6372796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6372934Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6373487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6373602Z return func(*args, **kwargs) 2025-12-04T09:50:27.6374102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6374246Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6374757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6374881Z return func(*args, **kwargs) 2025-12-04T09:50:27.6375377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6375513Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6376039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6376156Z return func(*args, **kwargs) 2025-12-04T09:50:27.6376660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6376798Z value_states = self.v(current_states) 2025-12-04T09:50:27.6376813Z 2025-12-04T09:50:27.6376948Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6377093Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6377273Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6377677Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6377797Z res = mod(**inputs) 2025-12-04T09:50:27.6378301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6378430Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6378939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6379059Z layer_outputs = layer_module( 2025-12-04T09:50:27.6379543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6379677Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6380187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6380314Z return func(*args, **kwargs) 2025-12-04T09:50:27.6380814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6380964Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6381477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6381591Z return func(*args, **kwargs) 2025-12-04T09:50:27.6382102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6382242Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6382801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6382928Z return func(*args, **kwargs) 2025-12-04T09:50:27.6383422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6383590Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6383604Z 2025-12-04T09:50:27.6383791Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6384185Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6384331Z res = mod(**inputs) 2025-12-04T09:50:27.6384842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6384963Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6385506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6385629Z layer_outputs = layer_module( 2025-12-04T09:50:27.6386107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6386237Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6386851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6386986Z return func(*args, **kwargs) 2025-12-04T09:50:27.6387670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6387822Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6388351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6388473Z return func(*args, **kwargs) 2025-12-04T09:50:27.6388998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 613, in forward 2025-12-04T09:50:27.6389248Z hidden_states = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:50:27.6389265Z 2025-12-04T09:50:27.6389455Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6389872Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6389987Z res = mod(**inputs) 2025-12-04T09:50:27.6390590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6390722Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6396319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6396458Z layer_outputs = layer_module( 2025-12-04T09:50:27.6396943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6397086Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6397626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6397749Z return func(*args, **kwargs) 2025-12-04T09:50:27.6398272Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6398415Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6398944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6399066Z return func(*args, **kwargs) 2025-12-04T09:50:27.6399577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6399738Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6400317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6400439Z return func(*args, **kwargs) 2025-12-04T09:50:27.6401388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6401609Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6401626Z 2025-12-04T09:50:27.6401819Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6402240Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6402394Z res = mod(**inputs) 2025-12-04T09:50:27.6402934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6403060Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6403622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6403759Z layer_outputs = layer_module( 2025-12-04T09:50:27.6404237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6404371Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6404910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6405028Z return func(*args, **kwargs) 2025-12-04T09:50:27.6405555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6405697Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6406218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6406349Z return func(*args, **kwargs) 2025-12-04T09:50:27.6406861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6407025Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6407548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6407666Z return func(*args, **kwargs) 2025-12-04T09:50:27.6408192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6408320Z key_states = self.k(current_states) 2025-12-04T09:50:27.6408335Z 2025-12-04T09:50:27.6408525Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6408946Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6409056Z res = mod(**inputs) 2025-12-04T09:50:27.6409589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6409716Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6410237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6410372Z layer_outputs = layer_module( 2025-12-04T09:50:27.6410851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6410990Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6411535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6411651Z return func(*args, **kwargs) 2025-12-04T09:50:27.6412176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6412323Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6412846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6413018Z return func(*args, **kwargs) 2025-12-04T09:50:27.6413632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6413788Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6414330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6414443Z return func(*args, **kwargs) 2025-12-04T09:50:27.6414988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6415119Z value_states = self.v(current_states) 2025-12-04T09:50:27.6415134Z 2025-12-04T09:50:27.6415268Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6415444Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6415631Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6416042Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6416152Z res = mod(**inputs) 2025-12-04T09:50:27.6416655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6416789Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6417291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6417407Z layer_outputs = layer_module( 2025-12-04T09:50:27.6417889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6418017Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6418536Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6418652Z return func(*args, **kwargs) 2025-12-04T09:50:27.6419149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6419292Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6419802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6419925Z return func(*args, **kwargs) 2025-12-04T09:50:27.6420426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6420571Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6421088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6421204Z return func(*args, **kwargs) 2025-12-04T09:50:27.6421704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6421840Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6421855Z 2025-12-04T09:50:27.6422039Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6422442Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6422549Z res = mod(**inputs) 2025-12-04T09:50:27.6423054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6423193Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6423695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6423811Z layer_outputs = layer_module( 2025-12-04T09:50:27.6424296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6424461Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6424977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6425105Z return func(*args, **kwargs) 2025-12-04T09:50:27.6425596Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6425791Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6426287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6426514Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6427305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.6427490Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.6427507Z 2025-12-04T09:50:27.6427706Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6428118Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6428224Z res = mod(**inputs) 2025-12-04T09:50:27.6428751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6428882Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6429398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6429530Z layer_outputs = layer_module( 2025-12-04T09:50:27.6430011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6430154Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6430685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6430802Z return func(*args, **kwargs) 2025-12-04T09:50:27.6431322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6431478Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6431989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6432209Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6432730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.6432874Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.6432890Z 2025-12-04T09:50:27.6433078Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6433492Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6433608Z res = mod(**inputs) 2025-12-04T09:50:27.6434130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6434263Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6434783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6434910Z layer_outputs = layer_module( 2025-12-04T09:50:27.6435396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6435533Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6436061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6436183Z return func(*args, **kwargs) 2025-12-04T09:50:27.6436706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6436899Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6437419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6437627Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6438193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.6438329Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.6438344Z 2025-12-04T09:50:27.6438569Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6439088Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6439191Z res = mod(**inputs) 2025-12-04T09:50:27.6439727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6439851Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6440354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6440474Z layer_outputs = layer_module( 2025-12-04T09:50:27.6440939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6441081Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6441589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6441704Z return func(*args, **kwargs) 2025-12-04T09:50:27.6442211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6442346Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6442862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6442987Z return func(*args, **kwargs) 2025-12-04T09:50:27.6443487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6443636Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6444145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6444258Z return func(*args, **kwargs) 2025-12-04T09:50:27.6444770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6444895Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6444910Z 2025-12-04T09:50:27.6445102Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6445499Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6445602Z res = mod(**inputs) 2025-12-04T09:50:27.6446116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6446234Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6446736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6446865Z layer_outputs = layer_module( 2025-12-04T09:50:27.6447329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6447470Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6447982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6448094Z return func(*args, **kwargs) 2025-12-04T09:50:27.6448603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6448764Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6449271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6449392Z return func(*args, **kwargs) 2025-12-04T09:50:27.6449922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6450068Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6450603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6450716Z return func(*args, **kwargs) 2025-12-04T09:50:27.6451223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6451373Z key_states = self.k(current_states) 2025-12-04T09:50:27.6451390Z 2025-12-04T09:50:27.6451581Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6451974Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6452076Z res = mod(**inputs) 2025-12-04T09:50:27.6452587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6452727Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6458255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6458400Z layer_outputs = layer_module( 2025-12-04T09:50:27.6458882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6459026Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6459554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6459673Z return func(*args, **kwargs) 2025-12-04T09:50:27.6460192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6460330Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6460862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6460984Z return func(*args, **kwargs) 2025-12-04T09:50:27.6461497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6461645Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6462172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6462295Z return func(*args, **kwargs) 2025-12-04T09:50:27.6462812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6462949Z value_states = self.v(current_states) 2025-12-04T09:50:27.6462965Z 2025-12-04T09:50:27.6463110Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6463241Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6463430Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6463848Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6463956Z res = mod(**inputs) 2025-12-04T09:50:27.6464478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6464611Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6465135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6465264Z layer_outputs = layer_module( 2025-12-04T09:50:27.6465795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6465929Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6466463Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6466613Z return func(*args, **kwargs) 2025-12-04T09:50:27.6467242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6467430Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6467991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6468117Z return func(*args, **kwargs) 2025-12-04T09:50:27.6468661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6468807Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6469345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6469460Z return func(*args, **kwargs) 2025-12-04T09:50:27.6469982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6470110Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6470126Z 2025-12-04T09:50:27.6470316Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6470737Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6470842Z res = mod(**inputs) 2025-12-04T09:50:27.6471362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6471501Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6472019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6472153Z layer_outputs = layer_module( 2025-12-04T09:50:27.6472629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6472766Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6473303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6473419Z return func(*args, **kwargs) 2025-12-04T09:50:27.6473934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6474080Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6474608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6474738Z return func(*args, **kwargs) 2025-12-04T09:50:27.6475252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6475396Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6475924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6476044Z return func(*args, **kwargs) 2025-12-04T09:50:27.6476566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6476699Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6476714Z 2025-12-04T09:50:27.6476900Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6477318Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6477424Z res = mod(**inputs) 2025-12-04T09:50:27.6477971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6478104Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6478746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6478905Z layer_outputs = layer_module( 2025-12-04T09:50:27.6479368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6479501Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6480061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6480179Z return func(*args, **kwargs) 2025-12-04T09:50:27.6480705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6480851Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6481363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6481484Z return func(*args, **kwargs) 2025-12-04T09:50:27.6481983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6482127Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6482639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6482758Z return func(*args, **kwargs) 2025-12-04T09:50:27.6483263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6483387Z key_states = self.k(current_states) 2025-12-04T09:50:27.6483402Z 2025-12-04T09:50:27.6483584Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6483996Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6484104Z res = mod(**inputs) 2025-12-04T09:50:27.6484609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6484740Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6485247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6485376Z layer_outputs = layer_module( 2025-12-04T09:50:27.6485841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6485971Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6486496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6486617Z return func(*args, **kwargs) 2025-12-04T09:50:27.6487118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6487261Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6487767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6487891Z return func(*args, **kwargs) 2025-12-04T09:50:27.6488384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6488527Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6489044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6489153Z return func(*args, **kwargs) 2025-12-04T09:50:27.6489665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6489819Z value_states = self.v(current_states) 2025-12-04T09:50:27.6489834Z 2025-12-04T09:50:27.6489964Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6490097Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6490275Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6490700Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6490808Z res = mod(**inputs) 2025-12-04T09:50:27.6491335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6491463Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6491962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6492106Z layer_outputs = layer_module( 2025-12-04T09:50:27.6492577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6492705Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6493212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6493333Z return func(*args, **kwargs) 2025-12-04T09:50:27.6493832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6493973Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6494484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6494595Z return func(*args, **kwargs) 2025-12-04T09:50:27.6495094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6495232Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6495740Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6495857Z return func(*args, **kwargs) 2025-12-04T09:50:27.6496347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6496480Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6496495Z 2025-12-04T09:50:27.6496678Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6497073Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6497178Z res = mod(**inputs) 2025-12-04T09:50:27.6497678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6497804Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6498304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6498423Z layer_outputs = layer_module( 2025-12-04T09:50:27.6498891Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6499019Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6499526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6499643Z return func(*args, **kwargs) 2025-12-04T09:50:27.6500137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6500278Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6501340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6501475Z return func(*args, **kwargs) 2025-12-04T09:50:27.6502084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 652, in forward 2025-12-04T09:50:27.6502328Z layer_output = hidden_states + self.dropout(attention_output[0]) 2025-12-04T09:50:27.6502344Z 2025-12-04T09:50:27.6502537Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6502990Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6503101Z res = mod(**inputs) 2025-12-04T09:50:27.6503665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6503792Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6504311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6504471Z layer_outputs = layer_module( 2025-12-04T09:50:27.6504951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6505090Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6505615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6505731Z return func(*args, **kwargs) 2025-12-04T09:50:27.6506251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6506402Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6506999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6507218Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6507731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.6507872Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.6507888Z 2025-12-04T09:50:27.6508073Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6508478Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6508587Z res = mod(**inputs) 2025-12-04T09:50:27.6509104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6509235Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6509750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6509871Z layer_outputs = layer_module( 2025-12-04T09:50:27.6510357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6510491Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6511013Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6511135Z return func(*args, **kwargs) 2025-12-04T09:50:27.6511649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6511808Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6512320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6512527Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6513051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.6513184Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.6513199Z 2025-12-04T09:50:27.6513393Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6513800Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6513943Z res = mod(**inputs) 2025-12-04T09:50:27.6514467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6514589Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6515136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6515324Z layer_outputs = layer_module( 2025-12-04T09:50:27.6515938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6516083Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6516609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6516752Z return func(*args, **kwargs) 2025-12-04T09:50:27.6517281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6517432Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6517943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6518157Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6518665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.6518804Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.6518821Z 2025-12-04T09:50:27.6519007Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6519411Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6519526Z res = mod(**inputs) 2025-12-04T09:50:27.6520042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6520172Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6520686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6520806Z layer_outputs = layer_module( 2025-12-04T09:50:27.6521291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6521424Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6521952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6522074Z return func(*args, **kwargs) 2025-12-04T09:50:27.6522590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6522731Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6523260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6523374Z return func(*args, **kwargs) 2025-12-04T09:50:27.6523886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6524028Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6524556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6524670Z return func(*args, **kwargs) 2025-12-04T09:50:27.6525187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6525320Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6525336Z 2025-12-04T09:50:27.6525524Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6525961Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6526068Z res = mod(**inputs) 2025-12-04T09:50:27.6526585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6526712Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6527274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6527392Z layer_outputs = layer_module( 2025-12-04T09:50:27.6528015Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6528147Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6528688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6528811Z return func(*args, **kwargs) 2025-12-04T09:50:27.6529309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6529450Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6529957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6530075Z return func(*args, **kwargs) 2025-12-04T09:50:27.6530578Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6530713Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6531230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6531340Z return func(*args, **kwargs) 2025-12-04T09:50:27.6531838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6531970Z key_states = self.k(current_states) 2025-12-04T09:50:27.6531984Z 2025-12-04T09:50:27.6532163Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6532560Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6532666Z res = mod(**inputs) 2025-12-04T09:50:27.6533170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6533299Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6533802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6533917Z layer_outputs = layer_module( 2025-12-04T09:50:27.6534385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6534516Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6535025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6535148Z return func(*args, **kwargs) 2025-12-04T09:50:27.6535644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6535787Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6536292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6536401Z return func(*args, **kwargs) 2025-12-04T09:50:27.6536902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6537038Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6537555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6537702Z return func(*args, **kwargs) 2025-12-04T09:50:27.6538200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6538333Z value_states = self.v(current_states) 2025-12-04T09:50:27.6538348Z 2025-12-04T09:50:27.6538481Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6538640Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6538833Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6539231Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6539364Z res = mod(**inputs) 2025-12-04T09:50:27.6539871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6539991Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6540528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6540648Z layer_outputs = layer_module( 2025-12-04T09:50:27.6541110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6541243Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6541754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6541872Z return func(*args, **kwargs) 2025-12-04T09:50:27.6542372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6542508Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6543019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6543133Z return func(*args, **kwargs) 2025-12-04T09:50:27.6543627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6543769Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6544275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6544393Z return func(*args, **kwargs) 2025-12-04T09:50:27.6544886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6545007Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6545022Z 2025-12-04T09:50:27.6545206Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6545597Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6545701Z res = mod(**inputs) 2025-12-04T09:50:27.6546201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6546325Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6546920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6547039Z layer_outputs = layer_module( 2025-12-04T09:50:27.6547675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6547815Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6548344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6548469Z return func(*args, **kwargs) 2025-12-04T09:50:27.6548984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6549124Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6549651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6549803Z return func(*args, **kwargs) 2025-12-04T09:50:27.6550315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6550473Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6551031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6551152Z return func(*args, **kwargs) 2025-12-04T09:50:27.6551694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6551821Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6551838Z 2025-12-04T09:50:27.6552030Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6552461Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6552577Z res = mod(**inputs) 2025-12-04T09:50:27.6553094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6553218Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6553738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6553858Z layer_outputs = layer_module( 2025-12-04T09:50:27.6554332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6554473Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6555001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6555122Z return func(*args, **kwargs) 2025-12-04T09:50:27.6555633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6555773Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6556304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6556415Z return func(*args, **kwargs) 2025-12-04T09:50:27.6556925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6557084Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6557610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6557736Z return func(*args, **kwargs) 2025-12-04T09:50:27.6558257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6558388Z key_states = self.k(current_states) 2025-12-04T09:50:27.6558406Z 2025-12-04T09:50:27.6558603Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6559012Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6559123Z res = mod(**inputs) 2025-12-04T09:50:27.6559748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6559872Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6560381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6560499Z layer_outputs = layer_module( 2025-12-04T09:50:27.6560960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6561095Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6561605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6561755Z return func(*args, **kwargs) 2025-12-04T09:50:27.6562253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6562391Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6562942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6563058Z return func(*args, **kwargs) 2025-12-04T09:50:27.6563584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6563734Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6564249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6564401Z return func(*args, **kwargs) 2025-12-04T09:50:27.6564902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6565030Z value_states = self.v(current_states) 2025-12-04T09:50:27.6565045Z 2025-12-04T09:50:27.6565182Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6565313Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6565504Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6565898Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6565998Z res = mod(**inputs) 2025-12-04T09:50:27.6566510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6566632Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6567132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6567263Z layer_outputs = layer_module( 2025-12-04T09:50:27.6567730Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6567871Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6568386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6568506Z return func(*args, **kwargs) 2025-12-04T09:50:27.6569021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6569161Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6569671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6569795Z return func(*args, **kwargs) 2025-12-04T09:50:27.6570295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6570453Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6570965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6571079Z return func(*args, **kwargs) 2025-12-04T09:50:27.6571592Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6571716Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6571730Z 2025-12-04T09:50:27.6571923Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6572319Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6572422Z res = mod(**inputs) 2025-12-04T09:50:27.6572940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6573105Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6573608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6573738Z layer_outputs = layer_module( 2025-12-04T09:50:27.6574203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6574377Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6574888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6575030Z return func(*args, **kwargs) 2025-12-04T09:50:27.6575538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6575683Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6576203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6576414Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6576913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.6577049Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.6577066Z 2025-12-04T09:50:27.6577245Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6577636Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6577794Z res = mod(**inputs) 2025-12-04T09:50:27.6578598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6578736Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6579262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6579385Z layer_outputs = layer_module( 2025-12-04T09:50:27.6579869Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6580004Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6580530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6580654Z return func(*args, **kwargs) 2025-12-04T09:50:27.6581167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6581329Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6581846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6582054Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6582576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.6582714Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.6582730Z 2025-12-04T09:50:27.6582922Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6583336Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6583449Z res = mod(**inputs) 2025-12-04T09:50:27.6583976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6584100Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6584616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6584745Z layer_outputs = layer_module( 2025-12-04T09:50:27.6585227Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6585920Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6586452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6586574Z return func(*args, **kwargs) 2025-12-04T09:50:27.6587251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6587448Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6587964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6588217Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6588736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.6588913Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.6588930Z 2025-12-04T09:50:27.6589120Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6589530Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6589643Z res = mod(**inputs) 2025-12-04T09:50:27.6590163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6590303Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6590822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6590949Z layer_outputs = layer_module( 2025-12-04T09:50:27.6591434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6591566Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6592102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6592232Z return func(*args, **kwargs) 2025-12-04T09:50:27.6592741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6592904Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6593417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 344, in forward 2025-12-04T09:50:27.6593644Z hidden_states = hidden_states + self.dropout(forwarded_states) 2025-12-04T09:50:27.6593659Z 2025-12-04T09:50:27.6593855Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6594264Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6594378Z res = mod(**inputs) 2025-12-04T09:50:27.6594907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6595036Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6595568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6595693Z layer_outputs = layer_module( 2025-12-04T09:50:27.6596174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6596326Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6596857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6596988Z return func(*args, **kwargs) 2025-12-04T09:50:27.6597505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6597646Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6598194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6598350Z return func(*args, **kwargs) 2025-12-04T09:50:27.6598961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6599108Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6599644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6599768Z return func(*args, **kwargs) 2025-12-04T09:50:27.6600295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6600422Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6600438Z 2025-12-04T09:50:27.6600632Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6601656Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6601783Z res = mod(**inputs) 2025-12-04T09:50:27.6602313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6602443Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6602969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6603098Z layer_outputs = layer_module( 2025-12-04T09:50:27.6603581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6603730Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6604257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6604388Z return func(*args, **kwargs) 2025-12-04T09:50:27.6604904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6605047Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6605584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6605705Z return func(*args, **kwargs) 2025-12-04T09:50:27.6606220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6606378Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6606906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6607040Z return func(*args, **kwargs) 2025-12-04T09:50:27.6607555Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6607688Z key_states = self.k(current_states) 2025-12-04T09:50:27.6607705Z 2025-12-04T09:50:27.6607911Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6608323Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6608441Z res = mod(**inputs) 2025-12-04T09:50:27.6608954Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6609083Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6609612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6609736Z layer_outputs = layer_module( 2025-12-04T09:50:27.6610213Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6610361Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6610887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6611062Z return func(*args, **kwargs) 2025-12-04T09:50:27.6611576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6611717Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6612254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6612413Z return func(*args, **kwargs) 2025-12-04T09:50:27.6612965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6613123Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6613649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6613820Z return func(*args, **kwargs) 2025-12-04T09:50:27.6614444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6614576Z value_states = self.v(current_states) 2025-12-04T09:50:27.6614593Z 2025-12-04T09:50:27.6614739Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6614869Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6615069Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6615473Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6615579Z res = mod(**inputs) 2025-12-04T09:50:27.6616092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6616214Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6616722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6616855Z layer_outputs = layer_module( 2025-12-04T09:50:27.6617319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6617458Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6617968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6618084Z return func(*args, **kwargs) 2025-12-04T09:50:27.6618595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 687, in forward 2025-12-04T09:50:27.6618731Z self_attention_outputs = self.layer[0]( 2025-12-04T09:50:27.6619242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6619362Z return func(*args, **kwargs) 2025-12-04T09:50:27.6619859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 603, in forward 2025-12-04T09:50:27.6620003Z attention_output = self.SelfAttention( 2025-12-04T09:50:27.6620506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6620618Z return func(*args, **kwargs) 2025-12-04T09:50:27.6621129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6621262Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6621277Z 2025-12-04T09:50:27.6621468Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6621867Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6621977Z res = mod(**inputs) 2025-12-04T09:50:27.6622494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6622625Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6623171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6623306Z layer_outputs = layer_module( 2025-12-04T09:50:27.6623777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6623956Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6624470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6624586Z return func(*args, **kwargs) 2025-12-04T09:50:27.6625123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6625265Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6625804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6625933Z return func(*args, **kwargs) 2025-12-04T09:50:27.6626438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6626584Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6627195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6627486Z return func(*args, **kwargs) 2025-12-04T09:50:27.6628007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 492, in forward 2025-12-04T09:50:27.6628140Z query_states = self.q(hidden_states) 2025-12-04T09:50:27.6628156Z 2025-12-04T09:50:27.6628355Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6628764Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6628874Z res = mod(**inputs) 2025-12-04T09:50:27.6629406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6629532Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6630051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6630184Z layer_outputs = layer_module( 2025-12-04T09:50:27.6630661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6630802Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6631333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6631450Z return func(*args, **kwargs) 2025-12-04T09:50:27.6631971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6632119Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6632650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6632779Z return func(*args, **kwargs) 2025-12-04T09:50:27.6633294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6633456Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6633982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6634104Z return func(*args, **kwargs) 2025-12-04T09:50:27.6634635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 513, in forward 2025-12-04T09:50:27.6634766Z key_states = self.k(current_states) 2025-12-04T09:50:27.6634781Z 2025-12-04T09:50:27.6634989Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6635444Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6635552Z res = mod(**inputs) 2025-12-04T09:50:27.6636082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6636236Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6636750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6636881Z layer_outputs = layer_module( 2025-12-04T09:50:27.6637386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6637533Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6638092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6638210Z return func(*args, **kwargs) 2025-12-04T09:50:27.6638741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6638882Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6639406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6639538Z return func(*args, **kwargs) 2025-12-04T09:50:27.6640047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6640235Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6645757Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6645888Z return func(*args, **kwargs) 2025-12-04T09:50:27.6646418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 514, in forward 2025-12-04T09:50:27.6646556Z value_states = self.v(current_states) 2025-12-04T09:50:27.6646574Z 2025-12-04T09:50:27.6646721Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6646864Z cudagraph partition due to non gpu ops 2025-12-04T09:50:27.6647056Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6647483Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6647599Z res = mod(**inputs) 2025-12-04T09:50:27.6648121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6648259Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6648782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6648919Z layer_outputs = layer_module( 2025-12-04T09:50:27.6649406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6649549Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6650079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6650200Z return func(*args, **kwargs) 2025-12-04T09:50:27.6650720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 711, in forward 2025-12-04T09:50:27.6650872Z cross_attention_outputs = self.layer[1]( 2025-12-04T09:50:27.6651404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6651534Z return func(*args, **kwargs) 2025-12-04T09:50:27.6652051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 640, in forward 2025-12-04T09:50:27.6652201Z attention_output = self.EncDecAttention( 2025-12-04T09:50:27.6652787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6652906Z return func(*args, **kwargs) 2025-12-04T09:50:27.6653421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 572, in forward 2025-12-04T09:50:27.6653592Z attn_output = self.o(attn_output) 2025-12-04T09:50:27.6653609Z 2025-12-04T09:50:27.6653799Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6654256Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6654367Z res = mod(**inputs) 2025-12-04T09:50:27.6654886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6655019Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6655568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6655705Z layer_outputs = layer_module( 2025-12-04T09:50:27.6656185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6656320Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6656859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6656977Z return func(*args, **kwargs) 2025-12-04T09:50:27.6657493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6657654Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6658171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6658393Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6658907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 288, in forward 2025-12-04T09:50:27.6659041Z hidden_states = self.wi(hidden_states) 2025-12-04T09:50:27.6659056Z 2025-12-04T09:50:27.6659253Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6659665Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6659786Z res = mod(**inputs) 2025-12-04T09:50:27.6660307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6660429Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6660956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6661081Z layer_outputs = layer_module( 2025-12-04T09:50:27.6661564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6661717Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6662246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6662377Z return func(*args, **kwargs) 2025-12-04T09:50:27.6662898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6663055Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6663582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6663793Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6664314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 289, in forward 2025-12-04T09:50:27.6664497Z hidden_states = self.act(hidden_states) 2025-12-04T09:50:27.6664513Z 2025-12-04T09:50:27.6664701Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6665126Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6665238Z res = mod(**inputs) 2025-12-04T09:50:27.6665788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1764, in forward 2025-12-04T09:50:27.6665925Z decoder_outputs = self.decoder( 2025-12-04T09:50:27.6666488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1100, in forward 2025-12-04T09:50:27.6666623Z layer_outputs = layer_module( 2025-12-04T09:50:27.6667225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:50:27.6667399Z return super().__call__(*args, **kwargs) 2025-12-04T09:50:27.6667946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:50:27.6668070Z return func(*args, **kwargs) 2025-12-04T09:50:27.6668583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 737, in forward 2025-12-04T09:50:27.6668754Z hidden_states = self.layer[-1](hidden_states) 2025-12-04T09:50:27.6669271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 343, in forward 2025-12-04T09:50:27.6669485Z forwarded_states = self.DenseReluDense(forwarded_states) 2025-12-04T09:50:27.6669999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 297, in forward 2025-12-04T09:50:27.6670134Z hidden_states = self.wo(hidden_states) 2025-12-04T09:50:27.6670150Z 2025-12-04T09:50:27.6670349Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6670759Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6670879Z res = mod(**inputs) 2025-12-04T09:50:27.6671404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1793, in forward 2025-12-04T09:50:27.6671551Z lm_logits = self.lm_head(sequence_output) 2025-12-04T09:50:27.6671569Z 2025-12-04T09:50:27.6671762Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:50:27.6672169Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:50:27.6672280Z res = mod(**inputs) 2025-12-04T09:50:27.6672816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1800, in forward 2025-12-04T09:50:27.6673072Z loss = loss_fct(lm_logits.view(-1, lm_logits.size(-1)), labels.view(-1)) 2025-12-04T09:50:27.6673088Z 2025-12-04T09:50:33.2263559Z Compilation time (from dynamo_timed): 21.542170495 2025-12-04T09:50:33.2484524Z pass 2025-12-04T09:50:33.2485133Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:50:33.2486787Z TIMING: _recursive_pre_grad_passes:0.07901 _recursive_joint_graph_passes:0.96039 _recursive_post_grad_passes:0.08914 async_compile.wait:0.00649 code_gen:4.59155 inductor_compile:8.97319 backend_compile:17.12491 gc:0.00029 entire_frame_compile:21.54217 total_wall_time:21.54217 2025-12-04T09:50:33.2488724Z STATS: call_* op count: 810 | FakeTensorMode.__torch_dispatch__:21989 | FakeTensor.__torch_dispatch__:3647 | ProxyTorchDispatchMode.__torch_dispatch__:4640 2025-12-04T09:50:33.2489736Z Dynamo produced 1 graphs covering 810 ops with 0 graph breaks (0 unique) 2025-12-04T09:50:36.0851402Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:50:36.0853405Z import pynvml # type: ignore[import] 2025-12-04T09:50:41.0784195Z 2025-12-04T09:50:45.4787189Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:50:45.4787833Z loading model: 0it [00:04, ?it/s] 2025-12-04T09:50:45.4821537Z cpu eval TrOCRForCausalLM 2025-12-04T09:50:45.7375337Z WARNING:common:fp64 golden ref were not generated for TrOCRForCausalLM. Setting accuracy check to cosine 2025-12-04T09:50:45.8009753Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:50:46.1397039Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:50:46.4686405Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:51:01.9486430Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9487163Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9487591Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9488000Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9488412Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9488781Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9489162Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9489557Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9489933Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9490320Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9490698Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9491080Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9491458Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9491844Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9492237Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9492613Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9493003Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9493390Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9493760Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9494143Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9494525Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9494891Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9495352Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9496087Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9496748Z res = mod(**inputs) 2025-12-04T09:51:01.9497489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9498313Z outputs = self.model.decoder( 2025-12-04T09:51:01.9499105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9499923Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9500623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9502078Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9502903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9503719Z return func(*args, **kwargs) 2025-12-04T09:51:01.9504505Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9505435Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9506251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9507080Z return self.act(input) 2025-12-04T09:51:01.9507311Z 2025-12-04T09:51:01.9507462Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9507877Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9508396Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9508779Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9509183Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9509588Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9509976Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9510444Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9510847Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9511233Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9511634Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9512152Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9512890Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9513571Z res = mod(**inputs) 2025-12-04T09:51:01.9514393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9515241Z outputs = self.model.decoder( 2025-12-04T09:51:01.9516034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9516869Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9517594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9518467Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9519252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9520032Z return func(*args, **kwargs) 2025-12-04T09:51:01.9520805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9521693Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9522489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9527603Z return self.act(input) 2025-12-04T09:51:01.9527821Z 2025-12-04T09:51:01.9527981Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9528368Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9528773Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9529173Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9529550Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9529951Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9530344Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9530729Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9531130Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9531595Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9531991Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9532451Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9533188Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9533866Z res = mod(**inputs) 2025-12-04T09:51:01.9534633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9535482Z outputs = self.model.decoder( 2025-12-04T09:51:01.9536275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9537122Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9537859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9538622Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9539430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9540407Z return func(*args, **kwargs) 2025-12-04T09:51:01.9541176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9549946Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9550787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9551626Z return self.act(input) 2025-12-04T09:51:01.9551840Z 2025-12-04T09:51:01.9552004Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9552399Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9552840Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9553239Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9553620Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9554015Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9554447Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9554842Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9555224Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9555618Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9556012Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9556453Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9557203Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9557885Z res = mod(**inputs) 2025-12-04T09:51:01.9558636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9559597Z outputs = self.model.decoder( 2025-12-04T09:51:01.9560388Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9561201Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9561898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9562655Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9563437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9564218Z return func(*args, **kwargs) 2025-12-04T09:51:01.9564978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9565876Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9566671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9567359Z return self.act(input) 2025-12-04T09:51:01.9567579Z 2025-12-04T09:51:01.9567718Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9568112Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9568499Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9568876Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9569300Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9569687Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9570057Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9570446Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9570833Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9571202Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9571584Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9572018Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9572728Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9573378Z res = mod(**inputs) 2025-12-04T09:51:01.9574115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9574925Z outputs = self.model.decoder( 2025-12-04T09:51:01.9575733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9576543Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9577255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9578025Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9578806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9579587Z return func(*args, **kwargs) 2025-12-04T09:51:01.9580402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9581292Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9582113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9582827Z return self.act(input) 2025-12-04T09:51:01.9583029Z 2025-12-04T09:51:01.9583184Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9583562Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9583955Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9584337Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9584706Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9585088Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9585745Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9586295Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9586697Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9587189Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9587570Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9588028Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9588779Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9589459Z res = mod(**inputs) 2025-12-04T09:51:01.9590203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9591042Z outputs = self.model.decoder( 2025-12-04T09:51:01.9591848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9592674Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9593410Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9594177Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9594982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9595772Z return func(*args, **kwargs) 2025-12-04T09:51:01.9596571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9597499Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9598315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9599146Z return self.act(input) 2025-12-04T09:51:01.9599366Z 2025-12-04T09:51:01.9599505Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9599903Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9600278Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9600668Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9601701Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9602083Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9602478Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9602874Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9603269Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9603763Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9604159Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9604612Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9605342Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9606075Z res = mod(**inputs) 2025-12-04T09:51:01.9606839Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9607673Z outputs = self.model.decoder( 2025-12-04T09:51:01.9608547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9609389Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9610168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9610930Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9611739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9612544Z return func(*args, **kwargs) 2025-12-04T09:51:01.9613473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9614429Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9615224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9615936Z return self.act(input) 2025-12-04T09:51:01.9616138Z 2025-12-04T09:51:01.9616277Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9616672Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9617063Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9617438Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9617823Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9618210Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9618597Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9618963Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9619348Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9619731Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9620102Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9620540Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9621262Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9621904Z res = mod(**inputs) 2025-12-04T09:51:01.9622643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9623460Z outputs = self.model.decoder( 2025-12-04T09:51:01.9624250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9625045Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9625755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9626507Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9627559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9628365Z return func(*args, **kwargs) 2025-12-04T09:51:01.9629172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9630101Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9630901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9631630Z return self.act(input) 2025-12-04T09:51:01.9631883Z 2025-12-04T09:51:01.9632041Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9632442Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9632824Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9633219Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9633614Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9634028Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9634423Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9634816Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9635196Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9635623Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9636020Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9636458Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9637255Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9637928Z res = mod(**inputs) 2025-12-04T09:51:01.9638690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9639610Z outputs = self.model.decoder( 2025-12-04T09:51:01.9640393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9641209Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9641902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9642646Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9643427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9644199Z return func(*args, **kwargs) 2025-12-04T09:51:01.9644956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9645855Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9646649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9647355Z return self.act(input) 2025-12-04T09:51:01.9647558Z 2025-12-04T09:51:01.9647701Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9648166Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9648867Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9649314Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9649715Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9650113Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9650501Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9650878Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9651269Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9651663Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9652060Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9652498Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9653251Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9653926Z res = mod(**inputs) 2025-12-04T09:51:01.9654667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9655512Z outputs = self.model.decoder( 2025-12-04T09:51:01.9656316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9657146Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9657860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9658626Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9659439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9660408Z return func(*args, **kwargs) 2025-12-04T09:51:01.9661184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9662125Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9662919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9663608Z return self.act(input) 2025-12-04T09:51:01.9663835Z 2025-12-04T09:51:01.9664005Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9664395Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9664763Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9665150Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9665567Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9666000Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9666372Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9666839Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9667404Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9667789Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9668193Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9668650Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9669392Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9670048Z res = mod(**inputs) 2025-12-04T09:51:01.9670807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9671650Z outputs = self.model.decoder( 2025-12-04T09:51:01.9672446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9673288Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9674016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9674788Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9675577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9676387Z return func(*args, **kwargs) 2025-12-04T09:51:01.9677187Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9678101Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9679009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9679716Z return self.act(input) 2025-12-04T09:51:01.9679918Z 2025-12-04T09:51:01.9680073Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9680449Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9680834Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9681211Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9681580Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9681959Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9682339Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9682720Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9683086Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9683467Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9683854Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9684273Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9684989Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9685640Z res = mod(**inputs) 2025-12-04T09:51:01.9686357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 828, in forward 2025-12-04T09:51:01.9687214Z outputs = self.model.decoder( 2025-12-04T09:51:01.9687997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 652, in forward 2025-12-04T09:51:01.9688809Z layer_outputs = decoder_layer( 2025-12-04T09:51:01.9689533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:01.9690279Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:01.9691091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:01.9691857Z return func(*args, **kwargs) 2025-12-04T09:51:01.9692659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 407, in forward 2025-12-04T09:51:01.9693560Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:01.9694354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:01.9695037Z return self.act(input) 2025-12-04T09:51:01.9695251Z 2025-12-04T09:51:01.9695389Z cudagraph partition due to non gpu ops 2025-12-04T09:51:01.9695827Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9696531Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9697179Z res = mod(**inputs) 2025-12-04T09:51:01.9697910Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 844, in forward 2025-12-04T09:51:01.9698752Z logits = self.output_projection(outputs[0]) 2025-12-04T09:51:01.9699035Z 2025-12-04T09:51:01.9699221Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:01.9699938Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:01.9700592Z res = mod(**inputs) 2025-12-04T09:51:01.9701908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/trocr/modeling_trocr.py", line 849, in forward 2025-12-04T09:51:01.9702961Z loss = loss_fct(logits.view(-1, self.config.vocab_size), labels.view(-1)) 2025-12-04T09:51:01.9703395Z 2025-12-04T09:51:08.4368239Z Compilation time (from dynamo_timed): 20.923756209 2025-12-04T09:51:08.4422642Z pass 2025-12-04T09:51:08.4423324Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:51:08.4424994Z TIMING: _recursive_pre_grad_passes:0.05374 _recursive_joint_graph_passes:1.00582 _recursive_post_grad_passes:0.09036 async_compile.wait:1.08594 code_gen:6.2352 inductor_compile:10.24663 backend_compile:17.17484 gc:0.0013 entire_frame_compile:20.92376 total_wall_time:20.92376 2025-12-04T09:51:08.4427011Z STATS: call_* op count: 443 | FakeTensorMode.__torch_dispatch__:20105 | FakeTensor.__torch_dispatch__:3533 | ProxyTorchDispatchMode.__torch_dispatch__:3349 2025-12-04T09:51:08.4428249Z Dynamo produced 1 graphs covering 443 ops with 0 graph breaks (0 unique) 2025-12-04T09:51:11.2899064Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:51:11.2901434Z import pynvml # type: ignore[import] 2025-12-04T09:51:16.2407328Z 2025-12-04T09:51:27.9927717Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:51:27.9928189Z loading model: 0it [00:11, ?it/s] 2025-12-04T09:51:27.9960666Z cpu eval XGLMForCausalLM 2025-12-04T09:51:28.6425684Z WARNING:common:fp64 golden ref were not generated for XGLMForCausalLM. Setting accuracy check to cosine 2025-12-04T09:51:28.8081914Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:51:29.5339428Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:51:30.2551915Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:51:59.6029005Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6031009Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6031810Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6033015Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6034065Z res = mod(**inputs) 2025-12-04T09:51:59.6034844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6035679Z outputs = self.model( 2025-12-04T09:51:59.6036532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6037358Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6038093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6038865Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6039660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6040468Z return func(*args, **kwargs) 2025-12-04T09:51:59.6041259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6042141Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6042980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6043781Z return func(*args, **kwargs) 2025-12-04T09:51:59.6044564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6045462Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6045800Z 2025-12-04T09:51:59.6046118Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6046842Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6047499Z res = mod(**inputs) 2025-12-04T09:51:59.6048203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6048999Z outputs = self.model( 2025-12-04T09:51:59.6049737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6050537Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6051234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6051983Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6052765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6053543Z return func(*args, **kwargs) 2025-12-04T09:51:59.6054297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6055342Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6065767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6066570Z return func(*args, **kwargs) 2025-12-04T09:51:59.6067479Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6068325Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6068711Z 2025-12-04T09:51:59.6068930Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6069664Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6070342Z res = mod(**inputs) 2025-12-04T09:51:59.6071100Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6071944Z outputs = self.model( 2025-12-04T09:51:59.6072700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6073566Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6074305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6075059Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6075916Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6076723Z return func(*args, **kwargs) 2025-12-04T09:51:59.6077506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6078374Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6079228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6080034Z return func(*args, **kwargs) 2025-12-04T09:51:59.6080799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6081742Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6082156Z 2025-12-04T09:51:59.6082301Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6082767Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6083497Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6084178Z res = mod(**inputs) 2025-12-04T09:51:59.6084923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6085759Z outputs = self.model( 2025-12-04T09:51:59.6086504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6087327Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6088063Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6088815Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6089646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6090449Z return func(*args, **kwargs) 2025-12-04T09:51:59.6091234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6092099Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6092951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6093760Z return func(*args, **kwargs) 2025-12-04T09:51:59.6094532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6095376Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6095681Z 2025-12-04T09:51:59.6095876Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6096622Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6097287Z res = mod(**inputs) 2025-12-04T09:51:59.6098043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6098917Z outputs = self.model( 2025-12-04T09:51:59.6099676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6100521Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6101808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6102588Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6103480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6104286Z return func(*args, **kwargs) 2025-12-04T09:51:59.6105119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6105997Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6106942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6107746Z return func(*args, **kwargs) 2025-12-04T09:51:59.6108534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6109421Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6109735Z 2025-12-04T09:51:59.6109931Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6110679Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6111354Z res = mod(**inputs) 2025-12-04T09:51:59.6112083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6112899Z outputs = self.model( 2025-12-04T09:51:59.6113665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6114488Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6115203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6115982Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6116788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6117577Z return func(*args, **kwargs) 2025-12-04T09:51:59.6118482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6119408Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6120258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6121048Z return func(*args, **kwargs) 2025-12-04T09:51:59.6121825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6122755Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6123128Z 2025-12-04T09:51:59.6123295Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6123684Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6124132Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6124872Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6125529Z res = mod(**inputs) 2025-12-04T09:51:59.6126268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6127077Z outputs = self.model( 2025-12-04T09:51:59.6127850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6128750Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6129489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6130259Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6131116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6131916Z return func(*args, **kwargs) 2025-12-04T09:51:59.6132734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6133653Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6134486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6135212Z return self.act(input) 2025-12-04T09:51:59.6135428Z 2025-12-04T09:51:59.6135591Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6135984Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6136380Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6137563Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6138315Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6138976Z res = mod(**inputs) 2025-12-04T09:51:59.6139716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6140546Z outputs = self.model( 2025-12-04T09:51:59.6141295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6142119Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6142852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6143632Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6144423Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6145219Z return func(*args, **kwargs) 2025-12-04T09:51:59.6146002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6146980Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6147841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6148651Z return func(*args, **kwargs) 2025-12-04T09:51:59.6149441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6150339Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6150695Z 2025-12-04T09:51:59.6150895Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6151631Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6152296Z res = mod(**inputs) 2025-12-04T09:51:59.6153026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6153847Z outputs = self.model( 2025-12-04T09:51:59.6154605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6155425Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6156157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6156932Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6157734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6158595Z return func(*args, **kwargs) 2025-12-04T09:51:59.6159489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6160338Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6161197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6161955Z return func(*args, **kwargs) 2025-12-04T09:51:59.6162744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6163555Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6163818Z 2025-12-04T09:51:59.6164004Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6164746Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6165399Z res = mod(**inputs) 2025-12-04T09:51:59.6166119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6166891Z outputs = self.model( 2025-12-04T09:51:59.6167616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6168413Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6169102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6169848Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6170624Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6171391Z return func(*args, **kwargs) 2025-12-04T09:51:59.6172130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6172979Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6173798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6174572Z return func(*args, **kwargs) 2025-12-04T09:51:59.6175316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6176228Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6176603Z 2025-12-04T09:51:59.6176758Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6177182Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6177894Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6178545Z res = mod(**inputs) 2025-12-04T09:51:59.6179263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6180042Z outputs = self.model( 2025-12-04T09:51:59.6181122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6181995Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6182714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6183488Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6184297Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6185093Z return func(*args, **kwargs) 2025-12-04T09:51:59.6185859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6186821Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6187734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6188521Z return func(*args, **kwargs) 2025-12-04T09:51:59.6189307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6190185Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6190465Z 2025-12-04T09:51:59.6190669Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6191428Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6192100Z res = mod(**inputs) 2025-12-04T09:51:59.6192835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6193679Z outputs = self.model( 2025-12-04T09:51:59.6194424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6195246Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6195967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6196721Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6197522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6198325Z return func(*args, **kwargs) 2025-12-04T09:51:59.6199217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6200047Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6201439Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6202251Z return func(*args, **kwargs) 2025-12-04T09:51:59.6203012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6203876Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6204197Z 2025-12-04T09:51:59.6204387Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6205129Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6205781Z res = mod(**inputs) 2025-12-04T09:51:59.6206526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6207337Z outputs = self.model( 2025-12-04T09:51:59.6208091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6208898Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6209629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6210396Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6211181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6211986Z return func(*args, **kwargs) 2025-12-04T09:51:59.6212772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6213740Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6214545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6215318Z return func(*args, **kwargs) 2025-12-04T09:51:59.6216078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6217056Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6217431Z 2025-12-04T09:51:59.6217572Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6217973Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6218414Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6219164Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6219817Z res = mod(**inputs) 2025-12-04T09:51:59.6220540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6221367Z outputs = self.model( 2025-12-04T09:51:59.6222109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6222903Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6223657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6224391Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6225183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6225961Z return func(*args, **kwargs) 2025-12-04T09:51:59.6226837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6227902Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6228726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6229458Z return self.act(input) 2025-12-04T09:51:59.6229666Z 2025-12-04T09:51:59.6229812Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6230216Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6230614Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6231067Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6231788Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6232466Z res = mod(**inputs) 2025-12-04T09:51:59.6233216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6234025Z outputs = self.model( 2025-12-04T09:51:59.6234773Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6235595Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6236317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6237060Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6237863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6238784Z return func(*args, **kwargs) 2025-12-04T09:51:59.6239527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6240369Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6241190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6241960Z return func(*args, **kwargs) 2025-12-04T09:51:59.6242696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6243928Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6244275Z 2025-12-04T09:51:59.6244487Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6245217Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6245941Z res = mod(**inputs) 2025-12-04T09:51:59.6246685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6247499Z outputs = self.model( 2025-12-04T09:51:59.6248242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6249086Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6249809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6250607Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6251398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6252195Z return func(*args, **kwargs) 2025-12-04T09:51:59.6253030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6253887Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6254726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6255527Z return func(*args, **kwargs) 2025-12-04T09:51:59.6256419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6257226Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6257507Z 2025-12-04T09:51:59.6257699Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6258415Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6259051Z res = mod(**inputs) 2025-12-04T09:51:59.6259767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6260559Z outputs = self.model( 2025-12-04T09:51:59.6261289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6262068Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6262767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6263511Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6264289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6265043Z return func(*args, **kwargs) 2025-12-04T09:51:59.6265795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6266644Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6267724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6268524Z return func(*args, **kwargs) 2025-12-04T09:51:59.6269304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6270245Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6270632Z 2025-12-04T09:51:59.6270773Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6271222Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6271959Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6272617Z res = mod(**inputs) 2025-12-04T09:51:59.6273352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6274167Z outputs = self.model( 2025-12-04T09:51:59.6274917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6275779Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6276504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6277273Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6278098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6278896Z return func(*args, **kwargs) 2025-12-04T09:51:59.6281933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6282841Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6283698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6284481Z return func(*args, **kwargs) 2025-12-04T09:51:59.6285241Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6286052Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6286326Z 2025-12-04T09:51:59.6286514Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6287238Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6287887Z res = mod(**inputs) 2025-12-04T09:51:59.6288599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6289384Z outputs = self.model( 2025-12-04T09:51:59.6290116Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6290913Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6291604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6292356Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6293138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6293902Z return func(*args, **kwargs) 2025-12-04T09:51:59.6294663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6295511Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6296344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6297103Z return func(*args, **kwargs) 2025-12-04T09:51:59.6297865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6298710Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6299007Z 2025-12-04T09:51:59.6299207Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6299909Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6300563Z res = mod(**inputs) 2025-12-04T09:51:59.6301911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6302720Z outputs = self.model( 2025-12-04T09:51:59.6303483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6304309Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6305039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6310886Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6311836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6312636Z return func(*args, **kwargs) 2025-12-04T09:51:59.6313411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6314336Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6315182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6316008Z return func(*args, **kwargs) 2025-12-04T09:51:59.6316848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6317774Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6318160Z 2025-12-04T09:51:59.6318462Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6318857Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6319278Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6319998Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6320644Z res = mod(**inputs) 2025-12-04T09:51:59.6321348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6322137Z outputs = self.model( 2025-12-04T09:51:59.6322865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6323665Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6324357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6325108Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6325890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6326650Z return func(*args, **kwargs) 2025-12-04T09:51:59.6327406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6328290Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6329145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6329831Z return self.act(input) 2025-12-04T09:51:59.6330047Z 2025-12-04T09:51:59.6330188Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6330576Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6330942Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6331376Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6332098Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6346248Z res = mod(**inputs) 2025-12-04T09:51:59.6347403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6348248Z outputs = self.model( 2025-12-04T09:51:59.6349018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6349860Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6350576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6351350Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6352160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6352953Z return func(*args, **kwargs) 2025-12-04T09:51:59.6353748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6354713Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6355564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6356350Z return func(*args, **kwargs) 2025-12-04T09:51:59.6357191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6358088Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6358427Z 2025-12-04T09:51:59.6358664Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6359506Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6360161Z res = mod(**inputs) 2025-12-04T09:51:59.6360923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6361705Z outputs = self.model( 2025-12-04T09:51:59.6362442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6363243Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6363934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6364679Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6365460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6366241Z return func(*args, **kwargs) 2025-12-04T09:51:59.6366977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6367822Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6373744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6374554Z return func(*args, **kwargs) 2025-12-04T09:51:59.6375316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6376154Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6376431Z 2025-12-04T09:51:59.6376639Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6377366Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6378039Z res = mod(**inputs) 2025-12-04T09:51:59.6378791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6379682Z outputs = self.model( 2025-12-04T09:51:59.6380426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6381252Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6381991Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6382852Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6383636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6384418Z return func(*args, **kwargs) 2025-12-04T09:51:59.6385181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6386011Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6386927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6387904Z return func(*args, **kwargs) 2025-12-04T09:51:59.6388693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6389692Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6390102Z 2025-12-04T09:51:59.6390249Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6390713Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6391488Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6392162Z res = mod(**inputs) 2025-12-04T09:51:59.6392945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6393764Z outputs = self.model( 2025-12-04T09:51:59.6394506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6395360Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6396096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6396857Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6397666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6398576Z return func(*args, **kwargs) 2025-12-04T09:51:59.6399336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6400171Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6401582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6402400Z return func(*args, **kwargs) 2025-12-04T09:51:59.6403191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6404021Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6404322Z 2025-12-04T09:51:59.6404518Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6405262Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6405916Z res = mod(**inputs) 2025-12-04T09:51:59.6406659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6407475Z outputs = self.model( 2025-12-04T09:51:59.6408226Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6409032Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6409766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6410534Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6411327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6412125Z return func(*args, **kwargs) 2025-12-04T09:51:59.6412912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6413893Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6414698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6415474Z return func(*args, **kwargs) 2025-12-04T09:51:59.6416237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6417078Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6417376Z 2025-12-04T09:51:59.6417567Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6418286Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6419038Z res = mod(**inputs) 2025-12-04T09:51:59.6419744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6420540Z outputs = self.model( 2025-12-04T09:51:59.6421327Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6422124Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6422862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6423611Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6424397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6425201Z return func(*args, **kwargs) 2025-12-04T09:51:59.6425968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6426934Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6427953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6428741Z return func(*args, **kwargs) 2025-12-04T09:51:59.6429537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6430588Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6430993Z 2025-12-04T09:51:59.6431183Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6431573Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6432027Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6432771Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6433429Z res = mod(**inputs) 2025-12-04T09:51:59.6434181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6434996Z outputs = self.model( 2025-12-04T09:51:59.6435815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6436626Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6437354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6438130Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6438922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6439721Z return func(*args, **kwargs) 2025-12-04T09:51:59.6440509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6441417Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6442215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6442935Z return self.act(input) 2025-12-04T09:51:59.6443145Z 2025-12-04T09:51:59.6443305Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6443691Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6444084Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6444641Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6445356Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6445993Z res = mod(**inputs) 2025-12-04T09:51:59.6446716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6447570Z outputs = self.model( 2025-12-04T09:51:59.6448291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6449089Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6449804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6450596Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6451364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6452176Z return func(*args, **kwargs) 2025-12-04T09:51:59.6452936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6453769Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6454628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6455411Z return func(*args, **kwargs) 2025-12-04T09:51:59.6456170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6457020Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6457362Z 2025-12-04T09:51:59.6457551Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6458266Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6458908Z res = mod(**inputs) 2025-12-04T09:51:59.6459615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6460404Z outputs = self.model( 2025-12-04T09:51:59.6461134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6461929Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6462620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6463368Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6464159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6464939Z return func(*args, **kwargs) 2025-12-04T09:51:59.6465683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6466532Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6467635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6468421Z return func(*args, **kwargs) 2025-12-04T09:51:59.6469197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6470034Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6470307Z 2025-12-04T09:51:59.6470516Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6471243Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6471929Z res = mod(**inputs) 2025-12-04T09:51:59.6472675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6473477Z outputs = self.model( 2025-12-04T09:51:59.6474232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6475041Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6475759Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6476569Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6477351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6478131Z return func(*args, **kwargs) 2025-12-04T09:51:59.6478900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6479869Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6480701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6481458Z return func(*args, **kwargs) 2025-12-04T09:51:59.6482186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6483128Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6483520Z 2025-12-04T09:51:59.6483656Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6484079Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6484769Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6485402Z res = mod(**inputs) 2025-12-04T09:51:59.6486104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6486866Z outputs = self.model( 2025-12-04T09:51:59.6487586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6488371Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6489060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6489784Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6490548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6491301Z return func(*args, **kwargs) 2025-12-04T09:51:59.6492036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6492991Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6494054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6494836Z return func(*args, **kwargs) 2025-12-04T09:51:59.6495586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6496401Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6496689Z 2025-12-04T09:51:59.6496882Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6497602Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6498247Z res = mod(**inputs) 2025-12-04T09:51:59.6498972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6499776Z outputs = self.model( 2025-12-04T09:51:59.6500502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6501699Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6502422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6503177Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6503951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6504737Z return func(*args, **kwargs) 2025-12-04T09:51:59.6505589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6506447Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6507368Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6508205Z return func(*args, **kwargs) 2025-12-04T09:51:59.6508972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6509810Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6510163Z 2025-12-04T09:51:59.6510354Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6511084Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6511784Z res = mod(**inputs) 2025-12-04T09:51:59.6512513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6513328Z outputs = self.model( 2025-12-04T09:51:59.6514070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6514867Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6515598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6516355Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6517138Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6517906Z return func(*args, **kwargs) 2025-12-04T09:51:59.6518666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6519603Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6520400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6521142Z return func(*args, **kwargs) 2025-12-04T09:51:59.6521871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6522752Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6523103Z 2025-12-04T09:51:59.6523236Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6523646Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6524069Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6524760Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6525393Z res = mod(**inputs) 2025-12-04T09:51:59.6526106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6526885Z outputs = self.model( 2025-12-04T09:51:59.6527588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6528377Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6529060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6529805Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6530567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6531321Z return func(*args, **kwargs) 2025-12-04T09:51:59.6532049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6532896Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6533670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6534412Z return self.act(input) 2025-12-04T09:51:59.6534608Z 2025-12-04T09:51:59.6534755Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6535122Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6535490Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6535937Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6536632Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6537270Z res = mod(**inputs) 2025-12-04T09:51:59.6537994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6538769Z outputs = self.model( 2025-12-04T09:51:59.6539499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6540288Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6540976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6541696Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6542453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6543213Z return func(*args, **kwargs) 2025-12-04T09:51:59.6543957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6544785Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6545589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6546359Z return func(*args, **kwargs) 2025-12-04T09:51:59.6547393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6548265Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6548611Z 2025-12-04T09:51:59.6548803Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6549517Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6550162Z res = mod(**inputs) 2025-12-04T09:51:59.6550894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6551702Z outputs = self.model( 2025-12-04T09:51:59.6552446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6553245Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6553964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6554731Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6560482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6561280Z return func(*args, **kwargs) 2025-12-04T09:51:59.6562045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6562906Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6563731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6564519Z return func(*args, **kwargs) 2025-12-04T09:51:59.6565285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6566120Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6566387Z 2025-12-04T09:51:59.6566632Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6567353Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6568113Z res = mod(**inputs) 2025-12-04T09:51:59.6568802Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6569611Z outputs = self.model( 2025-12-04T09:51:59.6570329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6571142Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6571819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6572547Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6573362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6574112Z return func(*args, **kwargs) 2025-12-04T09:51:59.6574855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6575689Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6576500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6577251Z return func(*args, **kwargs) 2025-12-04T09:51:59.6577996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6578896Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6579269Z 2025-12-04T09:51:59.6579417Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6579833Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6580537Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6581177Z res = mod(**inputs) 2025-12-04T09:51:59.6581870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6582644Z outputs = self.model( 2025-12-04T09:51:59.6583364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6584147Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6584824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6585549Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6586320Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6587353Z return func(*args, **kwargs) 2025-12-04T09:51:59.6588130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6588991Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6589818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6590594Z return func(*args, **kwargs) 2025-12-04T09:51:59.6591372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6592204Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6592482Z 2025-12-04T09:51:59.6592682Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6593391Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6594051Z res = mod(**inputs) 2025-12-04T09:51:59.6594788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6595616Z outputs = self.model( 2025-12-04T09:51:59.6596364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6597173Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6597920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6598660Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6599579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6600349Z return func(*args, **kwargs) 2025-12-04T09:51:59.6601617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6602561Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6603398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6604171Z return func(*args, **kwargs) 2025-12-04T09:51:59.6604920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6605769Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6606069Z 2025-12-04T09:51:59.6606269Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6606987Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6607647Z res = mod(**inputs) 2025-12-04T09:51:59.6608372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6609175Z outputs = self.model( 2025-12-04T09:51:59.6609906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6610722Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6611442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6612199Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6612976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6613863Z return func(*args, **kwargs) 2025-12-04T09:51:59.6614607Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6615422Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6616229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6616996Z return func(*args, **kwargs) 2025-12-04T09:51:59.6617739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6623594Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6623976Z 2025-12-04T09:51:59.6624117Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6624537Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6624978Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6625720Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6626397Z res = mod(**inputs) 2025-12-04T09:51:59.6627267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6628076Z outputs = self.model( 2025-12-04T09:51:59.6628842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6629755Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6630470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6631239Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6632046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6632896Z return func(*args, **kwargs) 2025-12-04T09:51:59.6633705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6634627Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6635448Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6636203Z return self.act(input) 2025-12-04T09:51:59.6636414Z 2025-12-04T09:51:59.6636561Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6636963Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6637367Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6637802Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6638658Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6639316Z res = mod(**inputs) 2025-12-04T09:51:59.6640023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6640817Z outputs = self.model( 2025-12-04T09:51:59.6641552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6642348Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6643043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6643789Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6644568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6645345Z return func(*args, **kwargs) 2025-12-04T09:51:59.6646085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6646927Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6647748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6648501Z return func(*args, **kwargs) 2025-12-04T09:51:59.6649250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6650118Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6650447Z 2025-12-04T09:51:59.6650659Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6651361Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6652016Z res = mod(**inputs) 2025-12-04T09:51:59.6652737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6653515Z outputs = self.model( 2025-12-04T09:51:59.6654245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6655067Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6655772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6656500Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6657285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6658097Z return func(*args, **kwargs) 2025-12-04T09:51:59.6658854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6659682Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6660503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6661318Z return func(*args, **kwargs) 2025-12-04T09:51:59.6662089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6662902Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6663179Z 2025-12-04T09:51:59.6663368Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6664118Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6664757Z res = mod(**inputs) 2025-12-04T09:51:59.6665476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6666262Z outputs = self.model( 2025-12-04T09:51:59.6667087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6668086Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6668818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6669588Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6670376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6671172Z return func(*args, **kwargs) 2025-12-04T09:51:59.6671958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6672831Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6673658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6674456Z return func(*args, **kwargs) 2025-12-04T09:51:59.6675230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6676161Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6676559Z 2025-12-04T09:51:59.6676705Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6677154Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6677885Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6678540Z res = mod(**inputs) 2025-12-04T09:51:59.6679281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6680096Z outputs = self.model( 2025-12-04T09:51:59.6681031Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6681856Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6682589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6683359Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6684147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6684944Z return func(*args, **kwargs) 2025-12-04T09:51:59.6685723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6686584Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6687487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6688285Z return func(*args, **kwargs) 2025-12-04T09:51:59.6689062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6689927Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6690223Z 2025-12-04T09:51:59.6690417Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6691201Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6691874Z res = mod(**inputs) 2025-12-04T09:51:59.6692600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6693412Z outputs = self.model( 2025-12-04T09:51:59.6694196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6695006Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6695737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6696597Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6697386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6698140Z return func(*args, **kwargs) 2025-12-04T09:51:59.6698897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6699748Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6700556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6701903Z return func(*args, **kwargs) 2025-12-04T09:51:59.6702696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6703568Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6703874Z 2025-12-04T09:51:59.6704069Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6704807Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6705458Z res = mod(**inputs) 2025-12-04T09:51:59.6706179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6707044Z outputs = self.model( 2025-12-04T09:51:59.6707791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6708605Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6709302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6710065Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6710854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6711641Z return func(*args, **kwargs) 2025-12-04T09:51:59.6712400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6713260Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6714096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6714876Z return func(*args, **kwargs) 2025-12-04T09:51:59.6715641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6716561Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6717016Z 2025-12-04T09:51:59.6717161Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6717542Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6717982Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6718803Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6719492Z res = mod(**inputs) 2025-12-04T09:51:59.6720189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6721006Z outputs = self.model( 2025-12-04T09:51:59.6721736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6722508Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6723233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6723968Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6724735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6725485Z return func(*args, **kwargs) 2025-12-04T09:51:59.6726224Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6727093Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6727866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6728558Z return self.act(input) 2025-12-04T09:51:59.6728762Z 2025-12-04T09:51:59.6728896Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6729283Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6729650Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6730075Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6730785Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6731415Z res = mod(**inputs) 2025-12-04T09:51:59.6732122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6732898Z outputs = self.model( 2025-12-04T09:51:59.6733618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6734383Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6735062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6735792Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6736540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6737300Z return func(*args, **kwargs) 2025-12-04T09:51:59.6738039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6738881Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6739674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6740445Z return func(*args, **kwargs) 2025-12-04T09:51:59.6741196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6742039Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6742369Z 2025-12-04T09:51:59.6742556Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6743413Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6744365Z res = mod(**inputs) 2025-12-04T09:51:59.6745082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6745886Z outputs = self.model( 2025-12-04T09:51:59.6746628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6747572Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6748281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6749073Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6749867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6750638Z return func(*args, **kwargs) 2025-12-04T09:51:59.6751427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6752285Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6753125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6753902Z return func(*args, **kwargs) 2025-12-04T09:51:59.6754670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6755502Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6755770Z 2025-12-04T09:51:59.6755965Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6756689Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6757351Z res = mod(**inputs) 2025-12-04T09:51:59.6758089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6758976Z outputs = self.model( 2025-12-04T09:51:59.6759689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6760470Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6761163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6761889Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6762650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6763418Z return func(*args, **kwargs) 2025-12-04T09:51:59.6764156Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6764995Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6765803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6766564Z return func(*args, **kwargs) 2025-12-04T09:51:59.6767292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6768189Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6768561Z 2025-12-04T09:51:59.6768706Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6769131Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6769843Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6770487Z res = mod(**inputs) 2025-12-04T09:51:59.6771206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6771976Z outputs = self.model( 2025-12-04T09:51:59.6772701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6773521Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6774201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6774933Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6775732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6776497Z return func(*args, **kwargs) 2025-12-04T09:51:59.6777256Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6778088Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6778925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6779683Z return func(*args, **kwargs) 2025-12-04T09:51:59.6780414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6781224Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6781487Z 2025-12-04T09:51:59.6781688Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6782381Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6783019Z res = mod(**inputs) 2025-12-04T09:51:59.6783729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6784504Z outputs = self.model( 2025-12-04T09:51:59.6785212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6785992Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6786750Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6787753Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6788545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6789332Z return func(*args, **kwargs) 2025-12-04T09:51:59.6790098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6790949Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6791784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6792569Z return func(*args, **kwargs) 2025-12-04T09:51:59.6793336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6794171Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6794481Z 2025-12-04T09:51:59.6794669Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6795391Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6796028Z res = mod(**inputs) 2025-12-04T09:51:59.6796751Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6797552Z outputs = self.model( 2025-12-04T09:51:59.6798294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6799200Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6799894Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6800632Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6801955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6802897Z return func(*args, **kwargs) 2025-12-04T09:51:59.6803670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6804533Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6810402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6811253Z return func(*args, **kwargs) 2025-12-04T09:51:59.6812086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6813002Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6813368Z 2025-12-04T09:51:59.6813507Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6813938Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6814384Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6815095Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6815805Z res = mod(**inputs) 2025-12-04T09:51:59.6816534Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6817344Z outputs = self.model( 2025-12-04T09:51:59.6818167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6818949Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6819641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6820362Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6821131Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6821893Z return func(*args, **kwargs) 2025-12-04T09:51:59.6822643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6823498Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6824278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6824972Z return self.act(input) 2025-12-04T09:51:59.6825289Z 2025-12-04T09:51:59.6825439Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6825803Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6826173Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6826605Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6827564Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6828226Z res = mod(**inputs) 2025-12-04T09:51:59.6828955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6829742Z outputs = self.model( 2025-12-04T09:51:59.6830483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6831296Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6832016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6832768Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6833559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6834346Z return func(*args, **kwargs) 2025-12-04T09:51:59.6835105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6836014Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6836844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6837633Z return func(*args, **kwargs) 2025-12-04T09:51:59.6838400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6839416Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6839749Z 2025-12-04T09:51:59.6839928Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6840669Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6841296Z res = mod(**inputs) 2025-12-04T09:51:59.6842029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6842811Z outputs = self.model( 2025-12-04T09:51:59.6843520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6844310Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6845011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6845751Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6846513Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6847277Z return func(*args, **kwargs) 2025-12-04T09:51:59.6848025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6848839Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6849647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6850402Z return func(*args, **kwargs) 2025-12-04T09:51:59.6851142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6851923Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6852193Z 2025-12-04T09:51:59.6852375Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6853076Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6853714Z res = mod(**inputs) 2025-12-04T09:51:59.6854400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6855173Z outputs = self.model( 2025-12-04T09:51:59.6855710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6855849Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6856317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6856461Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6856971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6857090Z return func(*args, **kwargs) 2025-12-04T09:51:59.6857632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6857804Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6858318Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6858445Z return func(*args, **kwargs) 2025-12-04T09:51:59.6858974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6859258Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6859273Z 2025-12-04T09:51:59.6859404Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6859584Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6860025Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6860130Z res = mod(**inputs) 2025-12-04T09:51:59.6860655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6860810Z outputs = self.model( 2025-12-04T09:51:59.6861339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6861468Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6861966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6862100Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6862629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6862741Z return func(*args, **kwargs) 2025-12-04T09:51:59.6863288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6863458Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6863967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6864097Z return func(*args, **kwargs) 2025-12-04T09:51:59.6864640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6864781Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6864798Z 2025-12-04T09:51:59.6864992Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6865391Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6865507Z res = mod(**inputs) 2025-12-04T09:51:59.6866041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6866156Z outputs = self.model( 2025-12-04T09:51:59.6866796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6866938Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6867585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6872871Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6873415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6873543Z return func(*args, **kwargs) 2025-12-04T09:51:59.6874090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6874265Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6874803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6874922Z return func(*args, **kwargs) 2025-12-04T09:51:59.6875482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6875653Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6875669Z 2025-12-04T09:51:59.6875860Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6876284Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6876449Z res = mod(**inputs) 2025-12-04T09:51:59.6876994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6877118Z outputs = self.model( 2025-12-04T09:51:59.6877667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6877846Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6878336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6878514Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6879065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6879222Z return func(*args, **kwargs) 2025-12-04T09:51:59.6879787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6879968Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6880500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6880646Z return func(*args, **kwargs) 2025-12-04T09:51:59.6881193Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6881424Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6881440Z 2025-12-04T09:51:59.6881598Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6881740Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6881948Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6882366Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6882482Z res = mod(**inputs) 2025-12-04T09:51:59.6883046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6883166Z outputs = self.model( 2025-12-04T09:51:59.6883712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6883858Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6884446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6884599Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6885114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6885290Z return func(*args, **kwargs) 2025-12-04T09:51:59.6885913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6886162Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6886695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6886881Z return self.act(input) 2025-12-04T09:51:59.6886899Z 2025-12-04T09:51:59.6887050Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6887453Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6887622Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6887889Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6888329Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6888475Z res = mod(**inputs) 2025-12-04T09:51:59.6889126Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6889298Z outputs = self.model( 2025-12-04T09:51:59.6889904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6890111Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6890616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6890855Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6891445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6891617Z return func(*args, **kwargs) 2025-12-04T09:51:59.6892263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6892473Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6893112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6893248Z return func(*args, **kwargs) 2025-12-04T09:51:59.6893849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6905747Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6905794Z 2025-12-04T09:51:59.6906054Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6906492Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6906614Z res = mod(**inputs) 2025-12-04T09:51:59.6907378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6907514Z outputs = self.model( 2025-12-04T09:51:59.6908075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6908212Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6908711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6908848Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6909381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6909519Z return func(*args, **kwargs) 2025-12-04T09:51:59.6910062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6910256Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6910783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6910902Z return func(*args, **kwargs) 2025-12-04T09:51:59.6911460Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6911603Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6911620Z 2025-12-04T09:51:59.6911823Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6912238Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6912354Z res = mod(**inputs) 2025-12-04T09:51:59.6912913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6913033Z outputs = self.model( 2025-12-04T09:51:59.6913583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6913727Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6914211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6914362Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6915023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6915141Z return func(*args, **kwargs) 2025-12-04T09:51:59.6915702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6915926Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6916470Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6916636Z return func(*args, **kwargs) 2025-12-04T09:51:59.6917185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6917494Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6917511Z 2025-12-04T09:51:59.6917659Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6917848Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6918270Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6918378Z res = mod(**inputs) 2025-12-04T09:51:59.6919041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6919157Z outputs = self.model( 2025-12-04T09:51:59.6919685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6919823Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6920294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6920428Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6920955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6921074Z return func(*args, **kwargs) 2025-12-04T09:51:59.6921617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6921787Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6922303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6922432Z return func(*args, **kwargs) 2025-12-04T09:51:59.6922963Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6923116Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6923131Z 2025-12-04T09:51:59.6923317Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6923721Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6923841Z res = mod(**inputs) 2025-12-04T09:51:59.6924369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6924485Z outputs = self.model( 2025-12-04T09:51:59.6925023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6925147Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6925626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6925759Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6926270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6926398Z return func(*args, **kwargs) 2025-12-04T09:51:59.6926932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6927134Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6927662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6927778Z return func(*args, **kwargs) 2025-12-04T09:51:59.6928378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6928542Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6928557Z 2025-12-04T09:51:59.6928785Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6929195Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6929302Z res = mod(**inputs) 2025-12-04T09:51:59.6929871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6929988Z outputs = self.model( 2025-12-04T09:51:59.6930728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6930882Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6931605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6931751Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6932299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6932417Z return func(*args, **kwargs) 2025-12-04T09:51:59.6932976Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6933154Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6933682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6933819Z return func(*args, **kwargs) 2025-12-04T09:51:59.6934363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6934612Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6934629Z 2025-12-04T09:51:59.6934769Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6934908Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6935109Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6935524Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6935633Z res = mod(**inputs) 2025-12-04T09:51:59.6936191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6936311Z outputs = self.model( 2025-12-04T09:51:59.6936860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6936987Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6937467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6937617Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6938149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6938271Z return func(*args, **kwargs) 2025-12-04T09:51:59.6938828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6939045Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6939515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6939678Z return self.act(input) 2025-12-04T09:51:59.6939693Z 2025-12-04T09:51:59.6939831Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6939976Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6940112Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6940331Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6940755Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6940866Z res = mod(**inputs) 2025-12-04T09:51:59.6941449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6941568Z outputs = self.model( 2025-12-04T09:51:59.6942148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6942287Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6942771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6942914Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6943567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6943687Z return func(*args, **kwargs) 2025-12-04T09:51:59.6944223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6944399Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6944913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6945040Z return func(*args, **kwargs) 2025-12-04T09:51:59.6945572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6945777Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6945792Z 2025-12-04T09:51:59.6945975Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6946371Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6946488Z res = mod(**inputs) 2025-12-04T09:51:59.6947332Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6947552Z outputs = self.model( 2025-12-04T09:51:59.6948114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6948241Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6948737Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6948878Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6949411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6949544Z return func(*args, **kwargs) 2025-12-04T09:51:59.6950090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6950277Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6950804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6950927Z return func(*args, **kwargs) 2025-12-04T09:51:59.6951481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6951620Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6951639Z 2025-12-04T09:51:59.6951830Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6952289Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6952400Z res = mod(**inputs) 2025-12-04T09:51:59.6952958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6953104Z outputs = self.model( 2025-12-04T09:51:59.6953648Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6953785Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6954295Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6954432Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6955007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6955130Z return func(*args, **kwargs) 2025-12-04T09:51:59.6955689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6955867Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6956394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6956531Z return func(*args, **kwargs) 2025-12-04T09:51:59.6957076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6957340Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6957356Z 2025-12-04T09:51:59.6957496Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6957688Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6958116Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6958228Z res = mod(**inputs) 2025-12-04T09:51:59.6958882Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6959008Z outputs = self.model( 2025-12-04T09:51:59.6959543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6959684Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6960151Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6960285Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6960812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6960930Z return func(*args, **kwargs) 2025-12-04T09:51:59.6961459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6961640Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6962150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6962275Z return func(*args, **kwargs) 2025-12-04T09:51:59.6962809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.6962951Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.6962968Z 2025-12-04T09:51:59.6963165Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6963561Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6963678Z res = mod(**inputs) 2025-12-04T09:51:59.6964214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6964360Z outputs = self.model( 2025-12-04T09:51:59.6964898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6965020Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6965491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6965665Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6966185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6966336Z return func(*args, **kwargs) 2025-12-04T09:51:59.6966863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6967062Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6967588Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6967707Z return func(*args, **kwargs) 2025-12-04T09:51:59.6968250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.6968412Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.6968430Z 2025-12-04T09:51:59.6968612Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6969016Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6969122Z res = mod(**inputs) 2025-12-04T09:51:59.6969651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6969775Z outputs = self.model( 2025-12-04T09:51:59.6970310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6970451Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6970919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6971053Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6971583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6971699Z return func(*args, **kwargs) 2025-12-04T09:51:59.6972231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6972411Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6972919Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6973048Z return func(*args, **kwargs) 2025-12-04T09:51:59.6973572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.6973799Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.6973815Z 2025-12-04T09:51:59.6973958Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6974089Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6974287Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6974686Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6974792Z res = mod(**inputs) 2025-12-04T09:51:59.6975338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6975455Z outputs = self.model( 2025-12-04T09:51:59.6975985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6976150Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6976616Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6976761Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6977271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6977430Z return func(*args, **kwargs) 2025-12-04T09:51:59.6977971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.6978208Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.6978656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.6978783Z return self.act(input) 2025-12-04T09:51:59.6978797Z 2025-12-04T09:51:59.6978958Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6979106Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6979237Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6979421Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6979831Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6979941Z res = mod(**inputs) 2025-12-04T09:51:59.6980474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6980600Z outputs = self.model( 2025-12-04T09:51:59.6981133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6981267Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6981735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6981866Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6982394Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6982513Z return func(*args, **kwargs) 2025-12-04T09:51:59.6983052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6983227Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6983741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6983871Z return func(*args, **kwargs) 2025-12-04T09:51:59.6984400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.6984594Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.6984612Z 2025-12-04T09:51:59.6984803Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6985203Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6985318Z res = mod(**inputs) 2025-12-04T09:51:59.6985850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6985968Z outputs = self.model( 2025-12-04T09:51:59.6986506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6986632Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6987372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6987526Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6988061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6988226Z return func(*args, **kwargs) 2025-12-04T09:51:59.6988770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6988946Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6989482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6989636Z return func(*args, **kwargs) 2025-12-04T09:51:59.6990192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.6990364Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.6990380Z 2025-12-04T09:51:59.6990571Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6991021Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6991129Z res = mod(**inputs) 2025-12-04T09:51:59.6991680Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6991810Z outputs = self.model( 2025-12-04T09:51:59.6992354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6992496Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6993115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6993256Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6993872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6994000Z return func(*args, **kwargs) 2025-12-04T09:51:59.6994548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.6994737Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.6995267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6995388Z return func(*args, **kwargs) 2025-12-04T09:51:59.6995939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.6996192Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.6996208Z 2025-12-04T09:51:59.6996356Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.6996546Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.6996959Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.6997077Z res = mod(**inputs) 2025-12-04T09:51:59.6997627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.6997752Z outputs = self.model( 2025-12-04T09:51:59.6998308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.6998435Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.6998927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.6999067Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.6999597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.6999725Z return func(*args, **kwargs) 2025-12-04T09:51:59.7000265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7000446Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7001375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7001587Z return func(*args, **kwargs) 2025-12-04T09:51:59.7002145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7002292Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7002356Z 2025-12-04T09:51:59.7002546Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7002969Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7003115Z res = mod(**inputs) 2025-12-04T09:51:59.7003678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7003793Z outputs = self.model( 2025-12-04T09:51:59.7004374Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7004513Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7004998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7005136Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7005674Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7005795Z return func(*args, **kwargs) 2025-12-04T09:51:59.7006354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7006527Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7007050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7007187Z return func(*args, **kwargs) 2025-12-04T09:51:59.7007727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7007909Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7007925Z 2025-12-04T09:51:59.7008111Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7008520Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7008644Z res = mod(**inputs) 2025-12-04T09:51:59.7009190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7009308Z outputs = self.model( 2025-12-04T09:51:59.7009866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7009989Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7010480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7010619Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7011145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7011272Z return func(*args, **kwargs) 2025-12-04T09:51:59.7011817Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7011991Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7012527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7012647Z return func(*args, **kwargs) 2025-12-04T09:51:59.7013304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7013532Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7013577Z 2025-12-04T09:51:59.7013707Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7013848Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7014032Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7014439Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7014572Z res = mod(**inputs) 2025-12-04T09:51:59.7015098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7015222Z outputs = self.model( 2025-12-04T09:51:59.7015783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7015905Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7016411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7016552Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7017069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7017187Z return func(*args, **kwargs) 2025-12-04T09:51:59.7017714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7017931Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7018372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7018488Z return self.act(input) 2025-12-04T09:51:59.7018513Z 2025-12-04T09:51:59.7018649Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7018783Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7018926Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7019113Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7019520Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7019640Z res = mod(**inputs) 2025-12-04T09:51:59.7020170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7020286Z outputs = self.model( 2025-12-04T09:51:59.7020822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7020943Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7021421Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7021553Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7022072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7022198Z return func(*args, **kwargs) 2025-12-04T09:51:59.7022726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7022907Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7023416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7023537Z return func(*args, **kwargs) 2025-12-04T09:51:59.7024074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7024267Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7024282Z 2025-12-04T09:51:59.7024464Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7024877Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7024982Z res = mod(**inputs) 2025-12-04T09:51:59.7025572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7025681Z outputs = self.model( 2025-12-04T09:51:59.7026208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7026372Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7026927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7027062Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7027838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7027956Z return func(*args, **kwargs) 2025-12-04T09:51:59.7028545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7028723Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7029253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7029384Z return func(*args, **kwargs) 2025-12-04T09:51:59.7029924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7030075Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7030091Z 2025-12-04T09:51:59.7030277Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7030686Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7030807Z res = mod(**inputs) 2025-12-04T09:51:59.7031351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7031465Z outputs = self.model( 2025-12-04T09:51:59.7032018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7032142Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7032626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7032766Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7033294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7033423Z return func(*args, **kwargs) 2025-12-04T09:51:59.7033967Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7034147Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7034679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7034802Z return func(*args, **kwargs) 2025-12-04T09:51:59.7035348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7035595Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7035611Z 2025-12-04T09:51:59.7035749Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7035946Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7036353Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7036470Z res = mod(**inputs) 2025-12-04T09:51:59.7037016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7037132Z outputs = self.model( 2025-12-04T09:51:59.7037687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7037845Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7038328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7038471Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7039109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7039261Z return func(*args, **kwargs) 2025-12-04T09:51:59.7039791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7039989Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7040508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7040652Z return func(*args, **kwargs) 2025-12-04T09:51:59.7041188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7041334Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7041348Z 2025-12-04T09:51:59.7041532Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7041934Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7042040Z res = mod(**inputs) 2025-12-04T09:51:59.7042565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7042689Z outputs = self.model( 2025-12-04T09:51:59.7043211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7043339Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7043809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7043938Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7044459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7044573Z return func(*args, **kwargs) 2025-12-04T09:51:59.7045097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7045277Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7045792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7045916Z return func(*args, **kwargs) 2025-12-04T09:51:59.7046442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7046609Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7046626Z 2025-12-04T09:51:59.7046814Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7047209Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7047330Z res = mod(**inputs) 2025-12-04T09:51:59.7047854Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7047971Z outputs = self.model( 2025-12-04T09:51:59.7048507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7048630Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7049095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7049239Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7049752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7049915Z return func(*args, **kwargs) 2025-12-04T09:51:59.7050441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7050609Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7051164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7051277Z return func(*args, **kwargs) 2025-12-04T09:51:59.7051849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7052074Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7052090Z 2025-12-04T09:51:59.7052221Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7052394Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7052578Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7052978Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7053095Z res = mod(**inputs) 2025-12-04T09:51:59.7053623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7053747Z outputs = self.model( 2025-12-04T09:51:59.7054271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7054391Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7054866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7055032Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7060562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7060694Z return func(*args, **kwargs) 2025-12-04T09:51:59.7061302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7061524Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7061981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7062099Z return self.act(input) 2025-12-04T09:51:59.7062114Z 2025-12-04T09:51:59.7062257Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7062390Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7062526Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7062722Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7063133Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7063249Z res = mod(**inputs) 2025-12-04T09:51:59.7063798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7063917Z outputs = self.model( 2025-12-04T09:51:59.7064472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7064600Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7065090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7065224Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7065755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7065880Z return func(*args, **kwargs) 2025-12-04T09:51:59.7066425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7066598Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7067294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7067416Z return func(*args, **kwargs) 2025-12-04T09:51:59.7067969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7068206Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7068221Z 2025-12-04T09:51:59.7068413Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7068867Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7068979Z res = mod(**inputs) 2025-12-04T09:51:59.7069525Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7069697Z outputs = self.model( 2025-12-04T09:51:59.7070248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7070380Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7070857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7070993Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7071531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7071647Z return func(*args, **kwargs) 2025-12-04T09:51:59.7072207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7072379Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7072908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7073038Z return func(*args, **kwargs) 2025-12-04T09:51:59.7073580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7073717Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7073732Z 2025-12-04T09:51:59.7073927Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7074338Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7074458Z res = mod(**inputs) 2025-12-04T09:51:59.7075002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7075119Z outputs = self.model( 2025-12-04T09:51:59.7075675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7075803Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7076292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7076441Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7076969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7077103Z return func(*args, **kwargs) 2025-12-04T09:51:59.7077651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7077823Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7078473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7078586Z return func(*args, **kwargs) 2025-12-04T09:51:59.7079121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7079391Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7079406Z 2025-12-04T09:51:59.7079536Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7079729Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7080126Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7080258Z res = mod(**inputs) 2025-12-04T09:51:59.7080793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7080934Z outputs = self.model( 2025-12-04T09:51:59.7081466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7081583Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7082075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7082213Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7082719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7082835Z return func(*args, **kwargs) 2025-12-04T09:51:59.7083365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7083533Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7084048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7084160Z return func(*args, **kwargs) 2025-12-04T09:51:59.7084685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7084835Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7084850Z 2025-12-04T09:51:59.7085035Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7085436Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7085543Z res = mod(**inputs) 2025-12-04T09:51:59.7086070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7086190Z outputs = self.model( 2025-12-04T09:51:59.7086718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7086841Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7087316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7087446Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7087971Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7088088Z return func(*args, **kwargs) 2025-12-04T09:51:59.7088617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7088794Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7089304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7089422Z return func(*args, **kwargs) 2025-12-04T09:51:59.7089951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7090113Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7090128Z 2025-12-04T09:51:59.7090315Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7090712Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7090847Z res = mod(**inputs) 2025-12-04T09:51:59.7091381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7091493Z outputs = self.model( 2025-12-04T09:51:59.7092029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7092176Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7092638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7092803Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7093344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7093457Z return func(*args, **kwargs) 2025-12-04T09:51:59.7094017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7094186Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7094700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7094811Z return func(*args, **kwargs) 2025-12-04T09:51:59.7095338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7095564Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7095579Z 2025-12-04T09:51:59.7095713Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7095852Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7096031Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7096426Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7096539Z res = mod(**inputs) 2025-12-04T09:51:59.7097072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7097183Z outputs = self.model( 2025-12-04T09:51:59.7097720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7097839Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7098309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7098442Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7098952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7099072Z return func(*args, **kwargs) 2025-12-04T09:51:59.7099601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7099807Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7100251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7100363Z return self.act(input) 2025-12-04T09:51:59.7100378Z 2025-12-04T09:51:59.7100517Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7100649Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7101312Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7101555Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7101993Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7102113Z res = mod(**inputs) 2025-12-04T09:51:59.7102676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7102793Z outputs = self.model( 2025-12-04T09:51:59.7103343Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7103552Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7104030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7104212Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7104742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7104872Z return func(*args, **kwargs) 2025-12-04T09:51:59.7105453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7105627Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7107338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7107478Z return func(*args, **kwargs) 2025-12-04T09:51:59.7108027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7108230Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7108247Z 2025-12-04T09:51:59.7108435Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7108853Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7108961Z res = mod(**inputs) 2025-12-04T09:51:59.7109509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7109633Z outputs = self.model( 2025-12-04T09:51:59.7110173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7110308Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7110792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7110925Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7111457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7111575Z return func(*args, **kwargs) 2025-12-04T09:51:59.7112118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7112292Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7112815Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7112942Z return func(*args, **kwargs) 2025-12-04T09:51:59.7113486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7113624Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7113639Z 2025-12-04T09:51:59.7113832Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7114240Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7114349Z res = mod(**inputs) 2025-12-04T09:51:59.7114902Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7115015Z outputs = self.model( 2025-12-04T09:51:59.7115575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7115699Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7116179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7116327Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7116889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7117015Z return func(*args, **kwargs) 2025-12-04T09:51:59.7117565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7122683Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7123292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7123408Z return func(*args, **kwargs) 2025-12-04T09:51:59.7124006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7124260Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7124277Z 2025-12-04T09:51:59.7124441Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7124644Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7125055Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7125161Z res = mod(**inputs) 2025-12-04T09:51:59.7125719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7125837Z outputs = self.model( 2025-12-04T09:51:59.7126380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7126517Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7126997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7127145Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7127676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7127792Z return func(*args, **kwargs) 2025-12-04T09:51:59.7128348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7128524Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7129062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7129182Z return func(*args, **kwargs) 2025-12-04T09:51:59.7129729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7129885Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7129900Z 2025-12-04T09:51:59.7130088Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7130501Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7130615Z res = mod(**inputs) 2025-12-04T09:51:59.7131159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7131281Z outputs = self.model( 2025-12-04T09:51:59.7131823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7131950Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7132435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7132573Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7133108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7133225Z return func(*args, **kwargs) 2025-12-04T09:51:59.7133871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7134102Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7134615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7134737Z return func(*args, **kwargs) 2025-12-04T09:51:59.7135307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7135471Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7135485Z 2025-12-04T09:51:59.7135705Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7136103Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7136211Z res = mod(**inputs) 2025-12-04T09:51:59.7136775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7136890Z outputs = self.model( 2025-12-04T09:51:59.7137420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7137549Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7138018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7138165Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7138682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7138799Z return func(*args, **kwargs) 2025-12-04T09:51:59.7139335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7139503Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7140028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7140147Z return func(*args, **kwargs) 2025-12-04T09:51:59.7140682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7140919Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7140936Z 2025-12-04T09:51:59.7141074Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7141209Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7141407Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7141809Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7141925Z res = mod(**inputs) 2025-12-04T09:51:59.7142457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7142571Z outputs = self.model( 2025-12-04T09:51:59.7143121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7143245Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7143714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7143856Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7144366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7144490Z return func(*args, **kwargs) 2025-12-04T09:51:59.7145024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7145229Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7145682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7145827Z return self.act(input) 2025-12-04T09:51:59.7145842Z 2025-12-04T09:51:59.7145978Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7146111Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7146244Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7146434Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7146962Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7147070Z res = mod(**inputs) 2025-12-04T09:51:59.7147944Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7148065Z outputs = self.model( 2025-12-04T09:51:59.7148628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7148786Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7149274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7149417Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7149946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7150069Z return func(*args, **kwargs) 2025-12-04T09:51:59.7150620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7150795Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7151331Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7151448Z return func(*args, **kwargs) 2025-12-04T09:51:59.7151995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7152200Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7152216Z 2025-12-04T09:51:59.7152405Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7152823Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7152929Z res = mod(**inputs) 2025-12-04T09:51:59.7153482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7153609Z outputs = self.model( 2025-12-04T09:51:59.7154153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7154280Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7154770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7154911Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7155451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7155569Z return func(*args, **kwargs) 2025-12-04T09:51:59.7156115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7156299Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7156825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7156947Z return func(*args, **kwargs) 2025-12-04T09:51:59.7157499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7157638Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7157654Z 2025-12-04T09:51:59.7157856Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7158298Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7158407Z res = mod(**inputs) 2025-12-04T09:51:59.7158962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7159185Z outputs = self.model( 2025-12-04T09:51:59.7159753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7159875Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7160369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7160514Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7161054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7161173Z return func(*args, **kwargs) 2025-12-04T09:51:59.7161715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7161883Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7162407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7162527Z return func(*args, **kwargs) 2025-12-04T09:51:59.7163060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7163310Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7163325Z 2025-12-04T09:51:59.7163459Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7163639Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7164047Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7164155Z res = mod(**inputs) 2025-12-04T09:51:59.7164696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7164810Z outputs = self.model( 2025-12-04T09:51:59.7165333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7165471Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7165939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7166083Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7166598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7166716Z return func(*args, **kwargs) 2025-12-04T09:51:59.7167255Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7167426Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7167933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7168059Z return func(*args, **kwargs) 2025-12-04T09:51:59.7168589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7168741Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7168756Z 2025-12-04T09:51:59.7168934Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7169328Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7169439Z res = mod(**inputs) 2025-12-04T09:51:59.7169975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7170116Z outputs = self.model( 2025-12-04T09:51:59.7170646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7170766Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7171239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7171413Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7171926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7172076Z return func(*args, **kwargs) 2025-12-04T09:51:59.7172603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7172781Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7173322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7173441Z return func(*args, **kwargs) 2025-12-04T09:51:59.7173983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7174148Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7174166Z 2025-12-04T09:51:59.7174349Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7174755Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7174863Z res = mod(**inputs) 2025-12-04T09:51:59.7175403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7175515Z outputs = self.model( 2025-12-04T09:51:59.7176047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7176183Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7176646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7176793Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7177304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7177421Z return func(*args, **kwargs) 2025-12-04T09:51:59.7177959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7178128Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7178643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7178767Z return func(*args, **kwargs) 2025-12-04T09:51:59.7179300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7179534Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7179549Z 2025-12-04T09:51:59.7179683Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7179818Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7180014Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7180552Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7180658Z res = mod(**inputs) 2025-12-04T09:51:59.7181475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7181590Z outputs = self.model( 2025-12-04T09:51:59.7182150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7182275Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7182794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7182939Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7183471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7183621Z return func(*args, **kwargs) 2025-12-04T09:51:59.7184177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7184417Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7184883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7185005Z return self.act(input) 2025-12-04T09:51:59.7185020Z 2025-12-04T09:51:59.7185186Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7185335Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7185470Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7185668Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7186085Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7186194Z res = mod(**inputs) 2025-12-04T09:51:59.7186849Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7186973Z outputs = self.model( 2025-12-04T09:51:59.7187519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7187649Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7188129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7188269Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7188799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7188917Z return func(*args, **kwargs) 2025-12-04T09:51:59.7189471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7189649Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7190177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7190306Z return func(*args, **kwargs) 2025-12-04T09:51:59.7190852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7191060Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7191076Z 2025-12-04T09:51:59.7191265Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7191675Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7191789Z res = mod(**inputs) 2025-12-04T09:51:59.7192337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7192464Z outputs = self.model( 2025-12-04T09:51:59.7193016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7193143Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7193634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7193769Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7194300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7194430Z return func(*args, **kwargs) 2025-12-04T09:51:59.7195009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7195199Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7195725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7195878Z return func(*args, **kwargs) 2025-12-04T09:51:59.7196435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7196577Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7196621Z 2025-12-04T09:51:59.7196822Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7197230Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7197369Z res = mod(**inputs) 2025-12-04T09:51:59.7197929Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7198050Z outputs = self.model( 2025-12-04T09:51:59.7198695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7198824Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7199289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7199434Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7199950Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7200064Z return func(*args, **kwargs) 2025-12-04T09:51:59.7200606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7201301Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7201850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7201998Z return func(*args, **kwargs) 2025-12-04T09:51:59.7202545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7202807Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7202822Z 2025-12-04T09:51:59.7202962Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7203149Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7203579Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7203686Z res = mod(**inputs) 2025-12-04T09:51:59.7204237Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7204371Z outputs = self.model( 2025-12-04T09:51:59.7204915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7205050Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7205532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7205671Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7206222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7206344Z return func(*args, **kwargs) 2025-12-04T09:51:59.7206898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7207079Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7207615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7207829Z return func(*args, **kwargs) 2025-12-04T09:51:59.7208376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7208523Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7208590Z 2025-12-04T09:51:59.7208781Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7209187Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7209300Z res = mod(**inputs) 2025-12-04T09:51:59.7209885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7210002Z outputs = self.model( 2025-12-04T09:51:59.7210590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7210720Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7211210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7211344Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7211874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7212003Z return func(*args, **kwargs) 2025-12-04T09:51:59.7212544Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7212719Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7213248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7213365Z return func(*args, **kwargs) 2025-12-04T09:51:59.7214025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7214190Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7214205Z 2025-12-04T09:51:59.7214386Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7214793Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7214903Z res = mod(**inputs) 2025-12-04T09:51:59.7215428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7215597Z outputs = self.model( 2025-12-04T09:51:59.7216132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7216938Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7217646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7218379Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7219183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7220123Z return func(*args, **kwargs) 2025-12-04T09:51:59.7220877Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7221702Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7222515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7223284Z return func(*args, **kwargs) 2025-12-04T09:51:59.7224028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7224911Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7225280Z 2025-12-04T09:51:59.7225533Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7225939Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7226368Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7227142Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7228084Z res = mod(**inputs) 2025-12-04T09:51:59.7228824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7229612Z outputs = self.model( 2025-12-04T09:51:59.7230384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7231199Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7231945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7232688Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7233481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7234269Z return func(*args, **kwargs) 2025-12-04T09:51:59.7235043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7235936Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7236741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7237461Z return self.act(input) 2025-12-04T09:51:59.7237669Z 2025-12-04T09:51:59.7237810Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7238204Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7238587Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7239127Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7239832Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7240467Z res = mod(**inputs) 2025-12-04T09:51:59.7241175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7241945Z outputs = self.model( 2025-12-04T09:51:59.7242670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7243804Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7244527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7245289Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7246083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7246863Z return func(*args, **kwargs) 2025-12-04T09:51:59.7247636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7248484Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7249305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7250094Z return func(*args, **kwargs) 2025-12-04T09:51:59.7250862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7251749Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7252085Z 2025-12-04T09:51:59.7252273Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7253007Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7253673Z res = mod(**inputs) 2025-12-04T09:51:59.7254390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7255243Z outputs = self.model( 2025-12-04T09:51:59.7256094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7256873Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7257594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7258324Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7259115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7259878Z return func(*args, **kwargs) 2025-12-04T09:51:59.7260635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7261464Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7262264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7263011Z return func(*args, **kwargs) 2025-12-04T09:51:59.7263747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7264543Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7264801Z 2025-12-04T09:51:59.7264994Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7265689Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7266323Z res = mod(**inputs) 2025-12-04T09:51:59.7267280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7268078Z outputs = self.model( 2025-12-04T09:51:59.7268812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7269619Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7270328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7271070Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7271867Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7272656Z return func(*args, **kwargs) 2025-12-04T09:51:59.7273420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7274269Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7275107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7275888Z return func(*args, **kwargs) 2025-12-04T09:51:59.7276643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7277571Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7277961Z 2025-12-04T09:51:59.7278100Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7278543Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7279360Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7279998Z res = mod(**inputs) 2025-12-04T09:51:59.7280710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7281478Z outputs = self.model( 2025-12-04T09:51:59.7282196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7283014Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7283703Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7284423Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7285188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7285982Z return func(*args, **kwargs) 2025-12-04T09:51:59.7286710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7287577Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7288381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7289139Z return func(*args, **kwargs) 2025-12-04T09:51:59.7289892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7290699Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7290978Z 2025-12-04T09:51:59.7291159Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7291860Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7292485Z res = mod(**inputs) 2025-12-04T09:51:59.7293189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7293968Z outputs = self.model( 2025-12-04T09:51:59.7294678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7295462Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7296154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7296880Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7297638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7298396Z return func(*args, **kwargs) 2025-12-04T09:51:59.7299134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7299951Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7301317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7302122Z return func(*args, **kwargs) 2025-12-04T09:51:59.7302890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7303733Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7304045Z 2025-12-04T09:51:59.7304235Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7304960Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7305717Z res = mod(**inputs) 2025-12-04T09:51:59.7306501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7307398Z outputs = self.model( 2025-12-04T09:51:59.7308140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7308940Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7309655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7310412Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7311261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7312142Z return func(*args, **kwargs) 2025-12-04T09:51:59.7312915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7313774Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7314594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7315422Z return func(*args, **kwargs) 2025-12-04T09:51:59.7316192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7317151Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7317517Z 2025-12-04T09:51:59.7317658Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7318045Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7318637Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7319349Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7319977Z res = mod(**inputs) 2025-12-04T09:51:59.7320687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7321466Z outputs = self.model( 2025-12-04T09:51:59.7322185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7322978Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7323670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7324400Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7325159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7325923Z return func(*args, **kwargs) 2025-12-04T09:51:59.7326662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7327518Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7328296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7328994Z return self.act(input) 2025-12-04T09:51:59.7329194Z 2025-12-04T09:51:59.7329334Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7329698Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7330068Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7330487Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7331177Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7331808Z res = mod(**inputs) 2025-12-04T09:51:59.7332515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7333290Z outputs = self.model( 2025-12-04T09:51:59.7333999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7334783Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7335473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7336188Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7336947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7337703Z return func(*args, **kwargs) 2025-12-04T09:51:59.7338438Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7339259Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7340099Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7340856Z return func(*args, **kwargs) 2025-12-04T09:51:59.7341584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7342470Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7342801Z 2025-12-04T09:51:59.7342980Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7343709Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7344333Z res = mod(**inputs) 2025-12-04T09:51:59.7345036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7345838Z outputs = self.model( 2025-12-04T09:51:59.7346557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7347606Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7348313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7349066Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7349858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7350644Z return func(*args, **kwargs) 2025-12-04T09:51:59.7351413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7352277Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7353102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7353890Z return func(*args, **kwargs) 2025-12-04T09:51:59.7354659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7355473Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7355756Z 2025-12-04T09:51:59.7355948Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7356684Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7357344Z res = mod(**inputs) 2025-12-04T09:51:59.7358065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7358873Z outputs = self.model( 2025-12-04T09:51:59.7359714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7360494Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7361175Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7361907Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7362663Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7363412Z return func(*args, **kwargs) 2025-12-04T09:51:59.7364157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7364977Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7365780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7366521Z return func(*args, **kwargs) 2025-12-04T09:51:59.7367264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7368309Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7368932Z 2025-12-04T09:51:59.7369079Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7369506Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7370336Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7371134Z res = mod(**inputs) 2025-12-04T09:51:59.7381966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7382766Z outputs = self.model( 2025-12-04T09:51:59.7383589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7384377Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7385105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7385843Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7386598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7387684Z return func(*args, **kwargs) 2025-12-04T09:51:59.7388461Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7389328Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7390157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7390949Z return func(*args, **kwargs) 2025-12-04T09:51:59.7391715Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7392539Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7392831Z 2025-12-04T09:51:59.7393022Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7393758Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7394424Z res = mod(**inputs) 2025-12-04T09:51:59.7395143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7395938Z outputs = self.model( 2025-12-04T09:51:59.7396676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7397478Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7398186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7399029Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7399796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7400548Z return func(*args, **kwargs) 2025-12-04T09:51:59.7401979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7402837Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7403668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7404449Z return func(*args, **kwargs) 2025-12-04T09:51:59.7405207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7406063Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7406366Z 2025-12-04T09:51:59.7406552Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7407277Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7407933Z res = mod(**inputs) 2025-12-04T09:51:59.7408755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7409544Z outputs = self.model( 2025-12-04T09:51:59.7410280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7411124Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7411825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7412578Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7413477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7414362Z return func(*args, **kwargs) 2025-12-04T09:51:59.7415140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7415980Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7416786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7417560Z return func(*args, **kwargs) 2025-12-04T09:51:59.7418285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7419162Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7419520Z 2025-12-04T09:51:59.7419664Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7420036Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7420456Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7421161Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7421804Z res = mod(**inputs) 2025-12-04T09:51:59.7422504Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7423282Z outputs = self.model( 2025-12-04T09:51:59.7423999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7424767Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7425467Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7426191Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7427049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7427993Z return func(*args, **kwargs) 2025-12-04T09:51:59.7428758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7429656Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7435506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7436216Z return self.act(input) 2025-12-04T09:51:59.7436428Z 2025-12-04T09:51:59.7436571Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7436961Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7437335Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7437772Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7438500Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7439146Z res = mod(**inputs) 2025-12-04T09:51:59.7439878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7440678Z outputs = self.model( 2025-12-04T09:51:59.7441426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7442289Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7443130Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7443857Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7444647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7445413Z return func(*args, **kwargs) 2025-12-04T09:51:59.7446190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7447023Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7447847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7448604Z return func(*args, **kwargs) 2025-12-04T09:51:59.7449354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7450206Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7450527Z 2025-12-04T09:51:59.7450710Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7451414Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7452043Z res = mod(**inputs) 2025-12-04T09:51:59.7452746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7453517Z outputs = self.model( 2025-12-04T09:51:59.7454233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7455021Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7455706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7456441Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7457202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7457947Z return func(*args, **kwargs) 2025-12-04T09:51:59.7458688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7459513Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7460316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7461065Z return func(*args, **kwargs) 2025-12-04T09:51:59.7461813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7462616Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7462880Z 2025-12-04T09:51:59.7463079Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7463772Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7464404Z res = mod(**inputs) 2025-12-04T09:51:59.7465109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7465875Z outputs = self.model( 2025-12-04T09:51:59.7466594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7467654Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7468363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7469103Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7469886Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7470706Z return func(*args, **kwargs) 2025-12-04T09:51:59.7471462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7472311Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7473172Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7473950Z return func(*args, **kwargs) 2025-12-04T09:51:59.7474735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7475662Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7476054Z 2025-12-04T09:51:59.7476222Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7476665Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7477387Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7478046Z res = mod(**inputs) 2025-12-04T09:51:59.7478782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7479663Z outputs = self.model( 2025-12-04T09:51:59.7480378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7481167Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7481856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7482574Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7483340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7484100Z return func(*args, **kwargs) 2025-12-04T09:51:59.7484832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7485665Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7486469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7487229Z return func(*args, **kwargs) 2025-12-04T09:51:59.7487959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7488759Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7489029Z 2025-12-04T09:51:59.7489232Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7489940Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7490573Z res = mod(**inputs) 2025-12-04T09:51:59.7491286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7492060Z outputs = self.model( 2025-12-04T09:51:59.7497803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7498673Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7499396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7500152Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7501321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7502118Z return func(*args, **kwargs) 2025-12-04T09:51:59.7502881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7503849Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7504675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7505460Z return func(*args, **kwargs) 2025-12-04T09:51:59.7506221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7507215Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7507530Z 2025-12-04T09:51:59.7507718Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7508494Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7509151Z res = mod(**inputs) 2025-12-04T09:51:59.7509905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7510708Z outputs = self.model( 2025-12-04T09:51:59.7511452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7512256Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7512958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7513710Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7514494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7515262Z return func(*args, **kwargs) 2025-12-04T09:51:59.7516019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7516870Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7517707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7518490Z return func(*args, **kwargs) 2025-12-04T09:51:59.7519353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7520234Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7520592Z 2025-12-04T09:51:59.7520734Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7521100Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7521517Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7522210Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7522827Z res = mod(**inputs) 2025-12-04T09:51:59.7523519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7524288Z outputs = self.model( 2025-12-04T09:51:59.7525000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7525777Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7526465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7527192Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7527939Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7528689Z return func(*args, **kwargs) 2025-12-04T09:51:59.7529427Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7530284Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7531045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7531731Z return self.act(input) 2025-12-04T09:51:59.7531965Z 2025-12-04T09:51:59.7532108Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7532466Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7532829Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7533246Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7533975Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7534594Z res = mod(**inputs) 2025-12-04T09:51:59.7535289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7536088Z outputs = self.model( 2025-12-04T09:51:59.7536791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7537571Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7538281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7539006Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7539752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7540508Z return func(*args, **kwargs) 2025-12-04T09:51:59.7541245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7542067Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7542870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7543619Z return func(*args, **kwargs) 2025-12-04T09:51:59.7544366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7545204Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7545537Z 2025-12-04T09:51:59.7545721Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7546424Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7547151Z res = mod(**inputs) 2025-12-04T09:51:59.7548027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7548825Z outputs = self.model( 2025-12-04T09:51:59.7549558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7550351Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7551061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7551807Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7552591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7553362Z return func(*args, **kwargs) 2025-12-04T09:51:59.7554122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7554971Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7555988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7556775Z return func(*args, **kwargs) 2025-12-04T09:51:59.7557541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7558365Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7558633Z 2025-12-04T09:51:59.7558825Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7559546Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7560244Z res = mod(**inputs) 2025-12-04T09:51:59.7561025Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7561816Z outputs = self.model( 2025-12-04T09:51:59.7562558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7563401Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7564124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7564864Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7565639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7566457Z return func(*args, **kwargs) 2025-12-04T09:51:59.7567206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7568149Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7568942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7569688Z return func(*args, **kwargs) 2025-12-04T09:51:59.7570418Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7571308Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7571675Z 2025-12-04T09:51:59.7571816Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7572229Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7572924Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7573551Z res = mod(**inputs) 2025-12-04T09:51:59.7574249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7575025Z outputs = self.model( 2025-12-04T09:51:59.7575742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7576522Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7577197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7577919Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7578682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7579433Z return func(*args, **kwargs) 2025-12-04T09:51:59.7580169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7580997Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7581793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7582546Z return func(*args, **kwargs) 2025-12-04T09:51:59.7583283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7584081Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7584349Z 2025-12-04T09:51:59.7584676Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7585367Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7586002Z res = mod(**inputs) 2025-12-04T09:51:59.7586774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7587745Z outputs = self.model( 2025-12-04T09:51:59.7588511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7589313Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7590019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7590792Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7591577Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7592354Z return func(*args, **kwargs) 2025-12-04T09:51:59.7593149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7593993Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7594864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7595655Z return func(*args, **kwargs) 2025-12-04T09:51:59.7596412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7597251Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7597552Z 2025-12-04T09:51:59.7597739Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7598454Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7599202Z res = mod(**inputs) 2025-12-04T09:51:59.7599904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7600670Z outputs = self.model( 2025-12-04T09:51:59.7601937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7602734Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7603442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7604187Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7604959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7605734Z return func(*args, **kwargs) 2025-12-04T09:51:59.7606486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7607335Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7608150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7608930Z return func(*args, **kwargs) 2025-12-04T09:51:59.7609485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7609723Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7609739Z 2025-12-04T09:51:59.7609872Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7610009Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7610196Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7610612Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7610728Z res = mod(**inputs) 2025-12-04T09:51:59.7611273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7611387Z outputs = self.model( 2025-12-04T09:51:59.7611937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7612063Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7612549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7612765Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7613301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7613434Z return func(*args, **kwargs) 2025-12-04T09:51:59.7614149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7614352Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7614837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7614951Z return self.act(input) 2025-12-04T09:51:59.7614966Z 2025-12-04T09:51:59.7615104Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7615272Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7615410Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7615600Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7615996Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7616102Z res = mod(**inputs) 2025-12-04T09:51:59.7616640Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7616752Z outputs = self.model( 2025-12-04T09:51:59.7617278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7617399Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7617974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7618113Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7618888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7619018Z return func(*args, **kwargs) 2025-12-04T09:51:59.7619564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7619740Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7620279Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7620396Z return func(*args, **kwargs) 2025-12-04T09:51:59.7620940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7621141Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7621158Z 2025-12-04T09:51:59.7621347Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7621768Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7621879Z res = mod(**inputs) 2025-12-04T09:51:59.7622432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7622561Z outputs = self.model( 2025-12-04T09:51:59.7623105Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7623245Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7623725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7623866Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7624400Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7624518Z return func(*args, **kwargs) 2025-12-04T09:51:59.7625057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7625274Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7625796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7625956Z return func(*args, **kwargs) 2025-12-04T09:51:59.7626498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7626634Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7626650Z 2025-12-04T09:51:59.7626971Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7627387Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7627495Z res = mod(**inputs) 2025-12-04T09:51:59.7628080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7628196Z outputs = self.model( 2025-12-04T09:51:59.7628749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7628869Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7629347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7629486Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7630011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7630131Z return func(*args, **kwargs) 2025-12-04T09:51:59.7630672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7630844Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7631377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7631492Z return func(*args, **kwargs) 2025-12-04T09:51:59.7632038Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7632297Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7632313Z 2025-12-04T09:51:59.7632451Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7632649Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7633060Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7633167Z res = mod(**inputs) 2025-12-04T09:51:59.7633724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7633841Z outputs = self.model( 2025-12-04T09:51:59.7634384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7634512Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7634994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7635139Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7635668Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7635787Z return func(*args, **kwargs) 2025-12-04T09:51:59.7636336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7636511Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7637045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7637193Z return func(*args, **kwargs) 2025-12-04T09:51:59.7637733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7637884Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7637899Z 2025-12-04T09:51:59.7638117Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7638631Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7638740Z res = mod(**inputs) 2025-12-04T09:51:59.7639296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7639411Z outputs = self.model( 2025-12-04T09:51:59.7639961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7640083Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7640557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7640688Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7641204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7641319Z return func(*args, **kwargs) 2025-12-04T09:51:59.7641841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7642013Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7642526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7642648Z return func(*args, **kwargs) 2025-12-04T09:51:59.7643189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7643355Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7643370Z 2025-12-04T09:51:59.7643560Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7643958Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7644069Z res = mod(**inputs) 2025-12-04T09:51:59.7644611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7644728Z outputs = self.model( 2025-12-04T09:51:59.7645253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7645383Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7645848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7645987Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7646499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7646615Z return func(*args, **kwargs) 2025-12-04T09:51:59.7647150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7647324Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7647842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7647960Z return func(*args, **kwargs) 2025-12-04T09:51:59.7648486Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7648720Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7648738Z 2025-12-04T09:51:59.7648875Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7649037Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7649238Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7649635Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7649745Z res = mod(**inputs) 2025-12-04T09:51:59.7650302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7650417Z outputs = self.model( 2025-12-04T09:51:59.7650996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7651119Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7651584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7651754Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7652268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7652390Z return func(*args, **kwargs) 2025-12-04T09:51:59.7652913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7653126Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7653576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7653691Z return self.act(input) 2025-12-04T09:51:59.7653706Z 2025-12-04T09:51:59.7653852Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7653987Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7654124Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7654319Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7654717Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7654828Z res = mod(**inputs) 2025-12-04T09:51:59.7655372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7655488Z outputs = self.model( 2025-12-04T09:51:59.7656030Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7656156Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7656623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7656775Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7657289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7657402Z return func(*args, **kwargs) 2025-12-04T09:51:59.7657942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7658115Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7658635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7658752Z return func(*args, **kwargs) 2025-12-04T09:51:59.7659288Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7659491Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7659506Z 2025-12-04T09:51:59.7659692Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7660104Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7660210Z res = mod(**inputs) 2025-12-04T09:51:59.7660741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7660889Z outputs = self.model( 2025-12-04T09:51:59.7661415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7661534Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7662002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7662161Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7662710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7662824Z return func(*args, **kwargs) 2025-12-04T09:51:59.7663355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7663559Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7664072Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7664187Z return func(*args, **kwargs) 2025-12-04T09:51:59.7664721Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7664859Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7664874Z 2025-12-04T09:51:59.7665059Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7665452Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7665558Z res = mod(**inputs) 2025-12-04T09:51:59.7666093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7666203Z outputs = self.model( 2025-12-04T09:51:59.7666828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7666953Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7667595Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7667751Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7668281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7668400Z return func(*args, **kwargs) 2025-12-04T09:51:59.7668961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7669136Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7669670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7669791Z return func(*args, **kwargs) 2025-12-04T09:51:59.7670337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7670591Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7670607Z 2025-12-04T09:51:59.7670746Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7670934Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7671356Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7671467Z res = mod(**inputs) 2025-12-04T09:51:59.7672027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7672142Z outputs = self.model( 2025-12-04T09:51:59.7672688Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7672827Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7673356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7673501Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7674036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7674188Z return func(*args, **kwargs) 2025-12-04T09:51:59.7674745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7674922Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7675476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7675606Z return func(*args, **kwargs) 2025-12-04T09:51:59.7676183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7676338Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7676353Z 2025-12-04T09:51:59.7676543Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7676952Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7677068Z res = mod(**inputs) 2025-12-04T09:51:59.7677615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7677731Z outputs = self.model( 2025-12-04T09:51:59.7678286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7678411Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7678901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7679038Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7679572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7679701Z return func(*args, **kwargs) 2025-12-04T09:51:59.7685258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7685458Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7685993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7686116Z return func(*args, **kwargs) 2025-12-04T09:51:59.7686672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7686841Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7686856Z 2025-12-04T09:51:59.7687047Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7687469Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7687580Z res = mod(**inputs) 2025-12-04T09:51:59.7688135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7688257Z outputs = self.model( 2025-12-04T09:51:59.7688800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7688933Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7689416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7689559Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7690091Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7690210Z return func(*args, **kwargs) 2025-12-04T09:51:59.7690808Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7690982Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7691507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7691667Z return func(*args, **kwargs) 2025-12-04T09:51:59.7692214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7692483Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7692499Z 2025-12-04T09:51:59.7692638Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7692773Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7692996Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7693408Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7693523Z res = mod(**inputs) 2025-12-04T09:51:59.7694084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7694202Z outputs = self.model( 2025-12-04T09:51:59.7694760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7694886Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7695371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7695516Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7696171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7696286Z return func(*args, **kwargs) 2025-12-04T09:51:59.7696818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7697028Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7697474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7697592Z return self.act(input) 2025-12-04T09:51:59.7697607Z 2025-12-04T09:51:59.7697736Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7697871Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7698002Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7698194Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7698593Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7698698Z res = mod(**inputs) 2025-12-04T09:51:59.7699236Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7699353Z outputs = self.model( 2025-12-04T09:51:59.7699880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7700009Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7700473Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7700611Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7701660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7701782Z return func(*args, **kwargs) 2025-12-04T09:51:59.7702335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7702512Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7703043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7703252Z return func(*args, **kwargs) 2025-12-04T09:51:59.7703801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 157, in forward 2025-12-04T09:51:59.7704008Z query_states = self.q_proj(hidden_states) * self.scaling 2025-12-04T09:51:59.7704079Z 2025-12-04T09:51:59.7704337Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7704752Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7704920Z res = mod(**inputs) 2025-12-04T09:51:59.7705469Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7705602Z outputs = self.model( 2025-12-04T09:51:59.7706184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7706317Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7706898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7707037Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7707568Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7707697Z return func(*args, **kwargs) 2025-12-04T09:51:59.7708250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7708437Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7708968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7709092Z return func(*args, **kwargs) 2025-12-04T09:51:59.7709650Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 177, in forward 2025-12-04T09:51:59.7709788Z key_states = self.k_proj(current_states) 2025-12-04T09:51:59.7709804Z 2025-12-04T09:51:59.7709999Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7710408Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7710524Z res = mod(**inputs) 2025-12-04T09:51:59.7711080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7711201Z outputs = self.model( 2025-12-04T09:51:59.7711743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7711881Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7712366Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7712517Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7713049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7713172Z return func(*args, **kwargs) 2025-12-04T09:51:59.7713735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7713914Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7714446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7714571Z return func(*args, **kwargs) 2025-12-04T09:51:59.7715114Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 199, in forward 2025-12-04T09:51:59.7715368Z attn_weights = torch.bmm(query_states, key_states.transpose(1, 2)) 2025-12-04T09:51:59.7715419Z 2025-12-04T09:51:59.7715560Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7715750Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7716170Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7716280Z res = mod(**inputs) 2025-12-04T09:51:59.7716858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7716981Z outputs = self.model( 2025-12-04T09:51:59.7717558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7717691Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7718170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7718333Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7718982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7719097Z return func(*args, **kwargs) 2025-12-04T09:51:59.7719632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7719801Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7720313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7720433Z return func(*args, **kwargs) 2025-12-04T09:51:59.7720966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 178, in forward 2025-12-04T09:51:59.7721104Z value_states = self.v_proj(current_states) 2025-12-04T09:51:59.7721130Z 2025-12-04T09:51:59.7721313Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7721711Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7721827Z res = mod(**inputs) 2025-12-04T09:51:59.7722356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7722469Z outputs = self.model( 2025-12-04T09:51:59.7723010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7723132Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7723610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7723742Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7724258Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7724389Z return func(*args, **kwargs) 2025-12-04T09:51:59.7724920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7725090Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7725610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7725727Z return func(*args, **kwargs) 2025-12-04T09:51:59.7726262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 245, in forward 2025-12-04T09:51:59.7726426Z attn_output = torch.bmm(attn_probs, value_states) 2025-12-04T09:51:59.7726441Z 2025-12-04T09:51:59.7726620Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7727025Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7727132Z res = mod(**inputs) 2025-12-04T09:51:59.7727657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7727810Z outputs = self.model( 2025-12-04T09:51:59.7728338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7731377Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7731923Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7732055Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7732610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7732728Z return func(*args, **kwargs) 2025-12-04T09:51:59.7733286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 333, in forward 2025-12-04T09:51:59.7733467Z hidden_states, self_attn_weights = self.self_attn( 2025-12-04T09:51:59.7733979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7734092Z return func(*args, **kwargs) 2025-12-04T09:51:59.7734623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 258, in forward 2025-12-04T09:51:59.7734880Z attn_output = attn_output.reshape(bsz, tgt_len, self.embed_dim) 2025-12-04T09:51:59.7734896Z 2025-12-04T09:51:59.7735036Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7735167Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7735353Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7735769Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7735876Z res = mod(**inputs) 2025-12-04T09:51:59.7736409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 668, in forward 2025-12-04T09:51:59.7736532Z outputs = self.model( 2025-12-04T09:51:59.7737060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 559, in forward 2025-12-04T09:51:59.7737191Z layer_outputs = decoder_layer( 2025-12-04T09:51:59.7737664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:51:59.7737797Z return super().__call__(*args, **kwargs) 2025-12-04T09:51:59.7738321Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func 2025-12-04T09:51:59.7738437Z return func(*args, **kwargs) 2025-12-04T09:51:59.7738961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 365, in forward 2025-12-04T09:51:59.7739180Z hidden_states = self.activation_fn(self.fc1(hidden_states)) 2025-12-04T09:51:59.7739621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:51:59.7739743Z return self.act(input) 2025-12-04T09:51:59.7739758Z 2025-12-04T09:51:59.7739888Z cudagraph partition due to non gpu ops 2025-12-04T09:51:59.7740071Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7740481Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7740587Z res = mod(**inputs) 2025-12-04T09:51:59.7741118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 685, in forward 2025-12-04T09:51:59.7741252Z logits = self.lm_head(outputs[0]) 2025-12-04T09:51:59.7741267Z 2025-12-04T09:51:59.7741449Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:51:59.7741854Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:51:59.7741960Z res = mod(**inputs) 2025-12-04T09:51:59.7742487Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xglm/modeling_xglm.py", line 689, in forward 2025-12-04T09:51:59.7742614Z loss = self.loss_function( 2025-12-04T09:51:59.7748201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 67, in ForCausalLMLoss 2025-12-04T09:51:59.7748712Z loss = fixed_cross_entropy(logits, shift_labels, num_items_in_batch, ignore_index, **kwargs) 2025-12-04T09:51:59.7749299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/loss/loss_utils.py", line 36, in fixed_cross_entropy 2025-12-04T09:51:59.7749689Z loss = nn.functional.cross_entropy(source, target, ignore_index=ignore_index, reduction=reduction) 2025-12-04T09:51:59.7749705Z 2025-12-04T09:52:10.8390128Z Compilation time (from dynamo_timed): 39.170049595 2025-12-04T09:52:10.8526656Z pass 2025-12-04T09:52:10.8527247Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:52:10.8528906Z TIMING: _recursive_pre_grad_passes:0.10633 _recursive_joint_graph_passes:1.66559 _recursive_post_grad_passes:0.35292 async_compile.wait:1.11967 code_gen:10.55774 inductor_compile:17.55411 backend_compile:32.06201 gc:0.00044 entire_frame_compile:39.17005 total_wall_time:39.17005 2025-12-04T09:52:10.8530919Z STATS: call_* op count: 921 | FakeTensorMode.__torch_dispatch__:44773 | FakeTensor.__torch_dispatch__:8380 | ProxyTorchDispatchMode.__torch_dispatch__:6458 2025-12-04T09:52:10.8531938Z Dynamo produced 1 graphs covering 921 ops with 0 graph breaks (0 unique) 2025-12-04T09:52:14.3170055Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:52:14.3171665Z import pynvml # type: ignore[import] 2025-12-04T09:52:19.2507549Z 2025-12-04T09:52:25.0211305Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:52:25.0211737Z loading model: 0it [00:05, ?it/s] 2025-12-04T09:52:25.0251154Z cpu eval XLNetLMHeadModel 2025-12-04T09:52:29.1837236Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:52:30.6438977Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:52:32.1018790Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:53:14.2779811Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2780641Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2781307Z res = mod(**inputs) 2025-12-04T09:53:14.2782062Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2782906Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2783729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1307, in forward 2025-12-04T09:53:14.2784566Z word_emb_k = self.word_embedding(input_ids) 2025-12-04T09:53:14.2784866Z 2025-12-04T09:53:14.2785060Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2785781Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2786436Z res = mod(**inputs) 2025-12-04T09:53:14.2787535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2788386Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2789352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1334, in forward 2025-12-04T09:53:14.2790293Z pos_emb = self.relative_positional_encoding(qlen, klen, bsz=bsz) 2025-12-04T09:53:14.2791353Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1157, in relative_positional_encoding 2025-12-04T09:53:14.2792403Z pos_emb = self.positional_embedding(fwd_pos_seq, inv_freq, bsz) 2025-12-04T09:53:14.2793856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1115, in positional_embedding 2025-12-04T09:53:14.2794936Z pos_emb = torch.cat([torch.sin(sinusoid_inp), torch.cos(sinusoid_inp)], dim=-1) 2025-12-04T09:53:14.2795438Z 2025-12-04T09:53:14.2795650Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2796372Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2797128Z res = mod(**inputs) 2025-12-04T09:53:14.2797893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2798734Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2799580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1334, in forward 2025-12-04T09:53:14.2800526Z pos_emb = self.relative_positional_encoding(qlen, klen, bsz=bsz) 2025-12-04T09:53:14.2802139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1157, in relative_positional_encoding 2025-12-04T09:53:14.2803192Z pos_emb = self.positional_embedding(fwd_pos_seq, inv_freq, bsz) 2025-12-04T09:53:14.2804219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1115, in positional_embedding 2025-12-04T09:53:14.2805304Z pos_emb = torch.cat([torch.sin(sinusoid_inp), torch.cos(sinusoid_inp)], dim=-1) 2025-12-04T09:53:14.2805743Z 2025-12-04T09:53:14.2805951Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2806677Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2807350Z res = mod(**inputs) 2025-12-04T09:53:14.2808104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2808977Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2809805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2810636Z outputs = layer_module( 2025-12-04T09:53:14.2811485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2815049Z outputs = self.rel_attn( 2025-12-04T09:53:14.2815855Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.2816743Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.2817053Z 2025-12-04T09:53:14.2817262Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2817982Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2818661Z res = mod(**inputs) 2025-12-04T09:53:14.2819472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2820317Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2821167Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2821995Z outputs = layer_module( 2025-12-04T09:53:14.2822771Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2823590Z outputs = self.rel_attn( 2025-12-04T09:53:14.2836095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.2837011Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.2837341Z 2025-12-04T09:53:14.2837711Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2838466Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2839135Z res = mod(**inputs) 2025-12-04T09:53:14.2839932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2840794Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2841689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2842507Z outputs = layer_module( 2025-12-04T09:53:14.2843285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2844112Z outputs = self.rel_attn( 2025-12-04T09:53:14.2844898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.2845720Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.2846569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.2847560Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.2847954Z 2025-12-04T09:53:14.2848164Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2848898Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2849569Z res = mod(**inputs) 2025-12-04T09:53:14.2850337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2851186Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2852039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1334, in forward 2025-12-04T09:53:14.2852989Z pos_emb = self.relative_positional_encoding(qlen, klen, bsz=bsz) 2025-12-04T09:53:14.2854047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1157, in relative_positional_encoding 2025-12-04T09:53:14.2855098Z pos_emb = self.positional_embedding(fwd_pos_seq, inv_freq, bsz) 2025-12-04T09:53:14.2856117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1115, in positional_embedding 2025-12-04T09:53:14.2857209Z pos_emb = torch.cat([torch.sin(sinusoid_inp), torch.cos(sinusoid_inp)], dim=-1) 2025-12-04T09:53:14.2857645Z 2025-12-04T09:53:14.2857853Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2858582Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2859259Z res = mod(**inputs) 2025-12-04T09:53:14.2860022Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2860897Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2861738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2862578Z outputs = layer_module( 2025-12-04T09:53:14.2863356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2864178Z outputs = self.rel_attn( 2025-12-04T09:53:14.2864958Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.2865909Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.2866301Z 2025-12-04T09:53:14.2866511Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2867352Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2868109Z res = mod(**inputs) 2025-12-04T09:53:14.2868874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2869771Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2870606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2871438Z outputs = layer_module( 2025-12-04T09:53:14.2872248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2873063Z outputs = self.rel_attn( 2025-12-04T09:53:14.2873909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.2874909Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.2875758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.2876789Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.2877187Z 2025-12-04T09:53:14.2877384Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2878123Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2878795Z res = mod(**inputs) 2025-12-04T09:53:14.2879539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2880394Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2881232Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2882039Z outputs = layer_module( 2025-12-04T09:53:14.2882818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2883647Z outputs = self.rel_attn( 2025-12-04T09:53:14.2884428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.2885301Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.2885627Z 2025-12-04T09:53:14.2885823Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2886563Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2887216Z res = mod(**inputs) 2025-12-04T09:53:14.2887977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2888844Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2889695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2890512Z outputs = layer_module( 2025-12-04T09:53:14.2891291Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2892124Z outputs = self.rel_attn( 2025-12-04T09:53:14.2892909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.2893738Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.2894584Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.2895549Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.2895915Z 2025-12-04T09:53:14.2896108Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2896848Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2897616Z res = mod(**inputs) 2025-12-04T09:53:14.2898385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2899269Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2900111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2901414Z outputs = layer_module( 2025-12-04T09:53:14.2902275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2903094Z outputs = self.rel_attn( 2025-12-04T09:53:14.2903881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.2904746Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.2905644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.2906606Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.2907079Z 2025-12-04T09:53:14.2907278Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2908017Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2908680Z res = mod(**inputs) 2025-12-04T09:53:14.2909446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2910311Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2911157Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2911975Z outputs = layer_module( 2025-12-04T09:53:14.2912755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2913588Z outputs = self.rel_attn( 2025-12-04T09:53:14.2914352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.2915207Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.2916096Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.2917045Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.2917380Z 2025-12-04T09:53:14.2917523Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.2917970Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2918707Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2919364Z res = mod(**inputs) 2025-12-04T09:53:14.2920121Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2920975Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2921818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2922629Z outputs = layer_module( 2025-12-04T09:53:14.2923509Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.2924611Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.2925725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.2926535Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.2927336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.2928260Z output_x = self.ff(output_x) 2025-12-04T09:53:14.2929069Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.2929896Z output = self.activation_function(output) 2025-12-04T09:53:14.2930613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.2931313Z return self.act(input) 2025-12-04T09:53:14.2931545Z 2025-12-04T09:53:14.2931683Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.2932120Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2932829Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2933460Z res = mod(**inputs) 2025-12-04T09:53:14.2934186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2935013Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2935827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2936989Z outputs = layer_module( 2025-12-04T09:53:14.2937770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2938600Z outputs = self.rel_attn( 2025-12-04T09:53:14.2939382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.2940239Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.2940560Z 2025-12-04T09:53:14.2940752Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2941485Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2942146Z res = mod(**inputs) 2025-12-04T09:53:14.2942899Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2943747Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2944581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2945386Z outputs = layer_module( 2025-12-04T09:53:14.2946155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2947060Z outputs = self.rel_attn( 2025-12-04T09:53:14.2947836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.2948702Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.2949032Z 2025-12-04T09:53:14.2949225Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2949963Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2950615Z res = mod(**inputs) 2025-12-04T09:53:14.2951378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2952229Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2953071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2953875Z outputs = layer_module( 2025-12-04T09:53:14.2954647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2955468Z outputs = self.rel_attn( 2025-12-04T09:53:14.2956239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.2957147Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.2957984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.2959113Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.2959489Z 2025-12-04T09:53:14.2959675Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2960421Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2961070Z res = mod(**inputs) 2025-12-04T09:53:14.2961805Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2962621Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2963435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2964243Z outputs = layer_module( 2025-12-04T09:53:14.2964982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2965782Z outputs = self.rel_attn( 2025-12-04T09:53:14.2966537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.2967457Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.2967838Z 2025-12-04T09:53:14.2968026Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2968739Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2969384Z res = mod(**inputs) 2025-12-04T09:53:14.2970104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2970920Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2971734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2972529Z outputs = layer_module( 2025-12-04T09:53:14.2973260Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2974057Z outputs = self.rel_attn( 2025-12-04T09:53:14.2974806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.2975604Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.2976398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.2977352Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.2977728Z 2025-12-04T09:53:14.2977922Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2978636Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2979271Z res = mod(**inputs) 2025-12-04T09:53:14.2980003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2980835Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2981643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2982439Z outputs = layer_module( 2025-12-04T09:53:14.2983185Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2983979Z outputs = self.rel_attn( 2025-12-04T09:53:14.2984712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.2985628Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.2985929Z 2025-12-04T09:53:14.2986125Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2986943Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2987810Z res = mod(**inputs) 2025-12-04T09:53:14.2988597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.2989446Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.2990271Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.2991090Z outputs = layer_module( 2025-12-04T09:53:14.2991857Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.2992680Z outputs = self.rel_attn( 2025-12-04T09:53:14.2993440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.2994262Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.2995098Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.2996047Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.2996419Z 2025-12-04T09:53:14.2996613Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.2997348Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.2998008Z res = mod(**inputs) 2025-12-04T09:53:14.2998816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3006049Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3006898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3007708Z outputs = layer_module( 2025-12-04T09:53:14.3008489Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3009319Z outputs = self.rel_attn( 2025-12-04T09:53:14.3010095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3010936Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3011825Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3012766Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3013212Z 2025-12-04T09:53:14.3013417Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3014113Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3014761Z res = mod(**inputs) 2025-12-04T09:53:14.3015496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3016312Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3017129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3017925Z outputs = layer_module( 2025-12-04T09:53:14.3018665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3019455Z outputs = self.rel_attn( 2025-12-04T09:53:14.3020208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3021177Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3022029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3022988Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3023330Z 2025-12-04T09:53:14.3023468Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3023900Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3024639Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3025282Z res = mod(**inputs) 2025-12-04T09:53:14.3026011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3026926Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3027964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3028789Z outputs = layer_module( 2025-12-04T09:53:14.3029563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3030693Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3031842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3032689Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3033506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3034329Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3035123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3035978Z output = self.activation_function(output) 2025-12-04T09:53:14.3036722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3037430Z return self.act(input) 2025-12-04T09:53:14.3037651Z 2025-12-04T09:53:14.3037795Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3038243Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3039077Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3039725Z res = mod(**inputs) 2025-12-04T09:53:14.3040453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3041280Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3042083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3042885Z outputs = layer_module( 2025-12-04T09:53:14.3043632Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3044417Z outputs = self.rel_attn( 2025-12-04T09:53:14.3045171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3046018Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3046313Z 2025-12-04T09:53:14.3046518Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3047227Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3047884Z res = mod(**inputs) 2025-12-04T09:53:14.3048613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3049534Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3050336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3051137Z outputs = layer_module( 2025-12-04T09:53:14.3051913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3052697Z outputs = self.rel_attn( 2025-12-04T09:53:14.3053481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3054334Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3054633Z 2025-12-04T09:53:14.3054830Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3055531Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3056175Z res = mod(**inputs) 2025-12-04T09:53:14.3056909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3057722Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3058537Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3059337Z outputs = layer_module( 2025-12-04T09:53:14.3060085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3060868Z outputs = self.rel_attn( 2025-12-04T09:53:14.3066042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3066960Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3067787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3068782Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3069250Z 2025-12-04T09:53:14.3069443Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3070983Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3071643Z res = mod(**inputs) 2025-12-04T09:53:14.3072403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3073262Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3074103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3074915Z outputs = layer_module( 2025-12-04T09:53:14.3075686Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3076513Z outputs = self.rel_attn( 2025-12-04T09:53:14.3077286Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3078228Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3078738Z 2025-12-04T09:53:14.3078924Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3079637Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3080269Z res = mod(**inputs) 2025-12-04T09:53:14.3080999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3081822Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3082633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3083413Z outputs = layer_module( 2025-12-04T09:53:14.3084261Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3085057Z outputs = self.rel_attn( 2025-12-04T09:53:14.3085832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3086638Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3087483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3088436Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3088809Z 2025-12-04T09:53:14.3088994Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3089705Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3090347Z res = mod(**inputs) 2025-12-04T09:53:14.3091083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3091893Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3092708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3093504Z outputs = layer_module( 2025-12-04T09:53:14.3094233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3095029Z outputs = self.rel_attn( 2025-12-04T09:53:14.3095785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3096640Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3096940Z 2025-12-04T09:53:14.3097125Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3097845Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3098491Z res = mod(**inputs) 2025-12-04T09:53:14.3099218Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3100044Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3101404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3102242Z outputs = layer_module( 2025-12-04T09:53:14.3103005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3103830Z outputs = self.rel_attn( 2025-12-04T09:53:14.3104604Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3105430Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3106265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3107324Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3107682Z 2025-12-04T09:53:14.3107896Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3108619Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3109289Z res = mod(**inputs) 2025-12-04T09:53:14.3110050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3110907Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3111732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3112564Z outputs = layer_module( 2025-12-04T09:53:14.3113471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3114298Z outputs = self.rel_attn( 2025-12-04T09:53:14.3115195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3116057Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3116998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3117940Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3118289Z 2025-12-04T09:53:14.3118589Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3119299Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3119952Z res = mod(**inputs) 2025-12-04T09:53:14.3120673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3121504Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3122328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3123118Z outputs = layer_module( 2025-12-04T09:53:14.3123972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3125135Z outputs = self.rel_attn( 2025-12-04T09:53:14.3125912Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3126754Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3127644Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3128589Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3128924Z 2025-12-04T09:53:14.3129076Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3129511Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3130244Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3130909Z res = mod(**inputs) 2025-12-04T09:53:14.3131646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3132494Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3133333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3134153Z outputs = layer_module( 2025-12-04T09:53:14.3134908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3136045Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3137194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3138043Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3138847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3139687Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3140599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3141412Z output = self.activation_function(output) 2025-12-04T09:53:14.3142127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3142912Z return self.act(input) 2025-12-04T09:53:14.3143115Z 2025-12-04T09:53:14.3143268Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3143696Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3144443Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3145092Z res = mod(**inputs) 2025-12-04T09:53:14.3145814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3146669Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3147765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3148593Z outputs = layer_module( 2025-12-04T09:53:14.3149355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3150179Z outputs = self.rel_attn( 2025-12-04T09:53:14.3150964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3151816Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3152128Z 2025-12-04T09:53:14.3152323Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3153053Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3153716Z res = mod(**inputs) 2025-12-04T09:53:14.3154449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3155299Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3156136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3156955Z outputs = layer_module( 2025-12-04T09:53:14.3157712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3158530Z outputs = self.rel_attn( 2025-12-04T09:53:14.3159424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3160259Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3160568Z 2025-12-04T09:53:14.3160751Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3161459Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3162104Z res = mod(**inputs) 2025-12-04T09:53:14.3162820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3163643Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3164457Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3165243Z outputs = layer_module( 2025-12-04T09:53:14.3165984Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3166778Z outputs = self.rel_attn( 2025-12-04T09:53:14.3167527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3168312Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3169123Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3170078Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3170449Z 2025-12-04T09:53:14.3170650Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3171346Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3172082Z res = mod(**inputs) 2025-12-04T09:53:14.3172813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3173658Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3174471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3175272Z outputs = layer_module( 2025-12-04T09:53:14.3176043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3176828Z outputs = self.rel_attn( 2025-12-04T09:53:14.3177582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3178508Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3178890Z 2025-12-04T09:53:14.3179088Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3179781Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3180425Z res = mod(**inputs) 2025-12-04T09:53:14.3181160Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3181979Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3182800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3183598Z outputs = layer_module( 2025-12-04T09:53:14.3184341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3185125Z outputs = self.rel_attn( 2025-12-04T09:53:14.3185878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3187084Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3187982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3188972Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3189375Z 2025-12-04T09:53:14.3189567Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3190305Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3190961Z res = mod(**inputs) 2025-12-04T09:53:14.3191717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3192567Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3193412Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3194234Z outputs = layer_module( 2025-12-04T09:53:14.3195011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3195844Z outputs = self.rel_attn( 2025-12-04T09:53:14.3196605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3197484Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3197809Z 2025-12-04T09:53:14.3197999Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3198846Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3199475Z res = mod(**inputs) 2025-12-04T09:53:14.3200207Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3201696Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3202526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3203421Z outputs = layer_module( 2025-12-04T09:53:14.3204192Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3205005Z outputs = self.rel_attn( 2025-12-04T09:53:14.3205807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3206638Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3207472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3208435Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3208799Z 2025-12-04T09:53:14.3208991Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3209723Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3210390Z res = mod(**inputs) 2025-12-04T09:53:14.3211143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3211992Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3212834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3213752Z outputs = layer_module( 2025-12-04T09:53:14.3214488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3215283Z outputs = self.rel_attn( 2025-12-04T09:53:14.3216033Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3216866Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3217719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3218638Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3218964Z 2025-12-04T09:53:14.3219162Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3219859Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3220504Z res = mod(**inputs) 2025-12-04T09:53:14.3221229Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3222062Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3222858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3223663Z outputs = layer_module( 2025-12-04T09:53:14.3224407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3225187Z outputs = self.rel_attn( 2025-12-04T09:53:14.3225942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3226850Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3227913Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3228848Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3229196Z 2025-12-04T09:53:14.3229337Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3229789Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3230626Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3231283Z res = mod(**inputs) 2025-12-04T09:53:14.3232067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3232917Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3233743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3234600Z outputs = layer_module( 2025-12-04T09:53:14.3235370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3236508Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3237638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3238488Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3239401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3240203Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3240957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3241778Z output = self.activation_function(output) 2025-12-04T09:53:14.3242495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3243184Z return self.act(input) 2025-12-04T09:53:14.3243400Z 2025-12-04T09:53:14.3243537Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3243975Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3244686Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3245325Z res = mod(**inputs) 2025-12-04T09:53:14.3246056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3246883Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3247687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3248481Z outputs = layer_module( 2025-12-04T09:53:14.3255372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3256200Z outputs = self.rel_attn( 2025-12-04T09:53:14.3256962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3257834Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3258143Z 2025-12-04T09:53:14.3258352Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3259067Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3259744Z res = mod(**inputs) 2025-12-04T09:53:14.3260501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3261357Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3262190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3263015Z outputs = layer_module( 2025-12-04T09:53:14.3263785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3264609Z outputs = self.rel_attn( 2025-12-04T09:53:14.3265377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3266392Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3266801Z 2025-12-04T09:53:14.3267012Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3267774Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3268441Z res = mod(**inputs) 2025-12-04T09:53:14.3269246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3270102Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3270930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3271758Z outputs = layer_module( 2025-12-04T09:53:14.3272530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3273352Z outputs = self.rel_attn( 2025-12-04T09:53:14.3274134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3274965Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3275803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3276774Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3277177Z 2025-12-04T09:53:14.3277370Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3278100Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3278868Z res = mod(**inputs) 2025-12-04T09:53:14.3279585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3280414Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3281223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3282004Z outputs = layer_module( 2025-12-04T09:53:14.3282749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3283545Z outputs = self.rel_attn( 2025-12-04T09:53:14.3284296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3285204Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3285592Z 2025-12-04T09:53:14.3285777Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3286483Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3287131Z res = mod(**inputs) 2025-12-04T09:53:14.3287848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3288686Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3289502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3290283Z outputs = layer_module( 2025-12-04T09:53:14.3291028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3291819Z outputs = self.rel_attn( 2025-12-04T09:53:14.3292571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3293360Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3294165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3295177Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3295548Z 2025-12-04T09:53:14.3295746Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3296475Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3297128Z res = mod(**inputs) 2025-12-04T09:53:14.3297888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3298701Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3299511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3300306Z outputs = layer_module( 2025-12-04T09:53:14.3301609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3302437Z outputs = self.rel_attn( 2025-12-04T09:53:14.3303216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3304101Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3304412Z 2025-12-04T09:53:14.3304603Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3305341Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3306011Z res = mod(**inputs) 2025-12-04T09:53:14.3306852Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3307702Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3308546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3309380Z outputs = layer_module( 2025-12-04T09:53:14.3310135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3310960Z outputs = self.rel_attn( 2025-12-04T09:53:14.3315377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3316265Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3317095Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3318051Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3318422Z 2025-12-04T09:53:14.3318613Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3319341Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3319991Z res = mod(**inputs) 2025-12-04T09:53:14.3320748Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3321599Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3322434Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3323254Z outputs = layer_module( 2025-12-04T09:53:14.3324125Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3324925Z outputs = self.rel_attn( 2025-12-04T09:53:14.3325662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3326489Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3327362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3329659Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3329988Z 2025-12-04T09:53:14.3330175Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3330938Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3331586Z res = mod(**inputs) 2025-12-04T09:53:14.3332309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3333187Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3334001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3334800Z outputs = layer_module( 2025-12-04T09:53:14.3335533Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3336335Z outputs = self.rel_attn( 2025-12-04T09:53:14.3337087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3337910Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3338781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3339702Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3340031Z 2025-12-04T09:53:14.3340187Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3340608Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3341322Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3341974Z res = mod(**inputs) 2025-12-04T09:53:14.3342710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3343526Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3344344Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3345146Z outputs = layer_module( 2025-12-04T09:53:14.3345876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3347263Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3348417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3349252Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3350044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3350873Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3351658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3352504Z output = self.activation_function(output) 2025-12-04T09:53:14.3353221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3353935Z return self.act(input) 2025-12-04T09:53:14.3354141Z 2025-12-04T09:53:14.3354285Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3354718Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3355449Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3356105Z res = mod(**inputs) 2025-12-04T09:53:14.3356851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3357753Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3358582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3359508Z outputs = layer_module( 2025-12-04T09:53:14.3360264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3361054Z outputs = self.rel_attn( 2025-12-04T09:53:14.3361816Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3362647Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3362934Z 2025-12-04T09:53:14.3363117Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3363813Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3364445Z res = mod(**inputs) 2025-12-04T09:53:14.3365162Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3365979Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3366780Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3367574Z outputs = layer_module( 2025-12-04T09:53:14.3368307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3369098Z outputs = self.rel_attn( 2025-12-04T09:53:14.3369844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3370683Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3370978Z 2025-12-04T09:53:14.3371158Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3371864Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3372498Z res = mod(**inputs) 2025-12-04T09:53:14.3373211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3374489Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3375350Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3376179Z outputs = layer_module( 2025-12-04T09:53:14.3376932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3377755Z outputs = self.rel_attn( 2025-12-04T09:53:14.3378526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3379401Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3380235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3381220Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3381607Z 2025-12-04T09:53:14.3381808Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3382524Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3383185Z res = mod(**inputs) 2025-12-04T09:53:14.3383937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3384773Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3385591Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3386411Z outputs = layer_module( 2025-12-04T09:53:14.3387360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3388172Z outputs = self.rel_attn( 2025-12-04T09:53:14.3389000Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3405362Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3405909Z 2025-12-04T09:53:14.3406260Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3407004Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3407657Z res = mod(**inputs) 2025-12-04T09:53:14.3408420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3409274Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3410119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3410927Z outputs = layer_module( 2025-12-04T09:53:14.3411698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3412510Z outputs = self.rel_attn( 2025-12-04T09:53:14.3413275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3414200Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3414999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3415945Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3416314Z 2025-12-04T09:53:14.3416494Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3417203Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3417841Z res = mod(**inputs) 2025-12-04T09:53:14.3418574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3419380Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3420188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3420975Z outputs = layer_module( 2025-12-04T09:53:14.3421698Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3422479Z outputs = self.rel_attn( 2025-12-04T09:53:14.3423220Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3424063Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3424368Z 2025-12-04T09:53:14.3424555Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3425272Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3425919Z res = mod(**inputs) 2025-12-04T09:53:14.3426638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3427742Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3428580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3429397Z outputs = layer_module( 2025-12-04T09:53:14.3430142Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3430963Z outputs = self.rel_attn( 2025-12-04T09:53:14.3431837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3432659Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3433524Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3434479Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3434835Z 2025-12-04T09:53:14.3435039Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3435780Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3436524Z res = mod(**inputs) 2025-12-04T09:53:14.3437380Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3438234Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3439061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3439887Z outputs = layer_module( 2025-12-04T09:53:14.3440655Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3441482Z outputs = self.rel_attn( 2025-12-04T09:53:14.3442240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3443087Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3443979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3444914Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3445262Z 2025-12-04T09:53:14.3445451Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3446176Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3446819Z res = mod(**inputs) 2025-12-04T09:53:14.3447545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3448499Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3449308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3450089Z outputs = layer_module( 2025-12-04T09:53:14.3450835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3451622Z outputs = self.rel_attn( 2025-12-04T09:53:14.3452371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3453185Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3454039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3454947Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3455273Z 2025-12-04T09:53:14.3455419Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3455837Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3456546Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3457179Z res = mod(**inputs) 2025-12-04T09:53:14.3457889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3458713Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3459519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3460403Z outputs = layer_module( 2025-12-04T09:53:14.3461127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3462255Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3463362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3464172Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3464974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3465775Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3466527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3467598Z output = self.activation_function(output) 2025-12-04T09:53:14.3468338Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3469051Z return self.act(input) 2025-12-04T09:53:14.3469251Z 2025-12-04T09:53:14.3469403Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3469830Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3470561Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3471217Z res = mod(**inputs) 2025-12-04T09:53:14.3471949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3472796Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3473628Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3474439Z outputs = layer_module( 2025-12-04T09:53:14.3475201Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3476013Z outputs = self.rel_attn( 2025-12-04T09:53:14.3476786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3477631Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3477939Z 2025-12-04T09:53:14.3478125Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3478847Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3479595Z res = mod(**inputs) 2025-12-04T09:53:14.3480303Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3481126Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3481933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3482736Z outputs = layer_module( 2025-12-04T09:53:14.3483466Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3484254Z outputs = self.rel_attn( 2025-12-04T09:53:14.3484999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3485831Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3486138Z 2025-12-04T09:53:14.3486323Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3487031Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3487669Z res = mod(**inputs) 2025-12-04T09:53:14.3488381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3489273Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3490084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3490910Z outputs = layer_module( 2025-12-04T09:53:14.3491641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3492431Z outputs = self.rel_attn( 2025-12-04T09:53:14.3493206Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3493986Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3494793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3495744Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3496118Z 2025-12-04T09:53:14.3496310Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3497007Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3497647Z res = mod(**inputs) 2025-12-04T09:53:14.3498369Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3506115Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3507092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3507917Z outputs = layer_module( 2025-12-04T09:53:14.3508691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3509495Z outputs = self.rel_attn( 2025-12-04T09:53:14.3510267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3511223Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3511617Z 2025-12-04T09:53:14.3511805Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3512537Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3513186Z res = mod(**inputs) 2025-12-04T09:53:14.3513930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3514776Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3515618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3516424Z outputs = layer_module( 2025-12-04T09:53:14.3517189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3518005Z outputs = self.rel_attn( 2025-12-04T09:53:14.3518861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3519651Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3520452Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3521395Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3521768Z 2025-12-04T09:53:14.3521954Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3522653Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3523293Z res = mod(**inputs) 2025-12-04T09:53:14.3524008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3524978Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3525786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3526632Z outputs = layer_module( 2025-12-04T09:53:14.3527364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3528148Z outputs = self.rel_attn( 2025-12-04T09:53:14.3528934Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3529767Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3530062Z 2025-12-04T09:53:14.3530240Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3530941Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3531582Z res = mod(**inputs) 2025-12-04T09:53:14.3532290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3533110Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3533920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3534708Z outputs = layer_module( 2025-12-04T09:53:14.3535440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3536233Z outputs = self.rel_attn( 2025-12-04T09:53:14.3536975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3537766Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3538551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3539480Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3539823Z 2025-12-04T09:53:14.3540013Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3540712Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3541350Z res = mod(**inputs) 2025-12-04T09:53:14.3542074Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3542886Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3543676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3544460Z outputs = layer_module( 2025-12-04T09:53:14.3545194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3545971Z outputs = self.rel_attn( 2025-12-04T09:53:14.3546783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3547790Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3548664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3549591Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3549937Z 2025-12-04T09:53:14.3550126Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3550851Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3551506Z res = mod(**inputs) 2025-12-04T09:53:14.3552239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3553150Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3553973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3554778Z outputs = layer_module( 2025-12-04T09:53:14.3555571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3556386Z outputs = self.rel_attn( 2025-12-04T09:53:14.3557176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3558010Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3558888Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3559923Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3560248Z 2025-12-04T09:53:14.3560390Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3560809Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3565355Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3566087Z res = mod(**inputs) 2025-12-04T09:53:14.3566830Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3567687Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3568526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3569342Z outputs = layer_module( 2025-12-04T09:53:14.3570093Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3571216Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3572354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3573190Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3574107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3574902Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3575654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3576458Z output = self.activation_function(output) 2025-12-04T09:53:14.3577164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3577849Z return self.act(input) 2025-12-04T09:53:14.3578044Z 2025-12-04T09:53:14.3578189Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3578606Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3579313Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3579948Z res = mod(**inputs) 2025-12-04T09:53:14.3580661Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3581479Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3582287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3583075Z outputs = layer_module( 2025-12-04T09:53:14.3583799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3584594Z outputs = self.rel_attn( 2025-12-04T09:53:14.3585334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3586246Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3586547Z 2025-12-04T09:53:14.3586803Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3587736Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3588391Z res = mod(**inputs) 2025-12-04T09:53:14.3589159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3589999Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3590824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3591633Z outputs = layer_module( 2025-12-04T09:53:14.3592377Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3593192Z outputs = self.rel_attn( 2025-12-04T09:53:14.3593955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3594814Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3595130Z 2025-12-04T09:53:14.3595315Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3596030Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3596678Z res = mod(**inputs) 2025-12-04T09:53:14.3597408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3598250Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3599073Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3599971Z outputs = layer_module( 2025-12-04T09:53:14.3600710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3602051Z outputs = self.rel_attn( 2025-12-04T09:53:14.3602834Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3603639Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3604471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3605452Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3605832Z 2025-12-04T09:53:14.3606030Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3606743Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3607410Z res = mod(**inputs) 2025-12-04T09:53:14.3608152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3608986Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3609822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3610635Z outputs = layer_module( 2025-12-04T09:53:14.3611392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3612193Z outputs = self.rel_attn( 2025-12-04T09:53:14.3612966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3614019Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3614396Z 2025-12-04T09:53:14.3614590Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3615391Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3616033Z res = mod(**inputs) 2025-12-04T09:53:14.3616789Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3617595Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3618446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3619237Z outputs = layer_module( 2025-12-04T09:53:14.3619974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3620757Z outputs = self.rel_attn( 2025-12-04T09:53:14.3621501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3622297Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3623084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3624363Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3624760Z 2025-12-04T09:53:14.3624946Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3625660Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3626304Z res = mod(**inputs) 2025-12-04T09:53:14.3627140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3627986Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3628907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3629907Z outputs = layer_module( 2025-12-04T09:53:14.3630681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3631491Z outputs = self.rel_attn( 2025-12-04T09:53:14.3632250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3633119Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3633433Z 2025-12-04T09:53:14.3633623Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3634338Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3634981Z res = mod(**inputs) 2025-12-04T09:53:14.3635718Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3636557Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3637378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3638192Z outputs = layer_module( 2025-12-04T09:53:14.3639052Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3639839Z outputs = self.rel_attn( 2025-12-04T09:53:14.3640575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3641368Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3642164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3643077Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3643424Z 2025-12-04T09:53:14.3643606Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3644420Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3645057Z res = mod(**inputs) 2025-12-04T09:53:14.3645798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3646618Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3647426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3647570Z outputs = layer_module( 2025-12-04T09:53:14.3648124Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3648241Z outputs = self.rel_attn( 2025-12-04T09:53:14.3648778Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3648934Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3649519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3649723Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3649740Z 2025-12-04T09:53:14.3649924Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3650323Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3650441Z res = mod(**inputs) 2025-12-04T09:53:14.3650990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3651128Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3651687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3651806Z outputs = layer_module( 2025-12-04T09:53:14.3652355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3652469Z outputs = self.rel_attn( 2025-12-04T09:53:14.3653012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3653162Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3653743Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3653950Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3653965Z 2025-12-04T09:53:14.3654098Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3654279Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3654684Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3654796Z res = mod(**inputs) 2025-12-04T09:53:14.3655347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3655492Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3656046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3656168Z outputs = layer_module( 2025-12-04T09:53:14.3656710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3657122Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3657687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3657815Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3658424Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3658541Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3659107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3659259Z output = self.activation_function(output) 2025-12-04T09:53:14.3659728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3659846Z return self.act(input) 2025-12-04T09:53:14.3659861Z 2025-12-04T09:53:14.3660007Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3660190Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3660595Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3660697Z res = mod(**inputs) 2025-12-04T09:53:14.3661250Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3661398Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3661940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3662054Z outputs = layer_module( 2025-12-04T09:53:14.3662606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3662721Z outputs = self.rel_attn( 2025-12-04T09:53:14.3663269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3663429Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3663444Z 2025-12-04T09:53:14.3663628Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3664043Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3664146Z res = mod(**inputs) 2025-12-04T09:53:14.3664700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3664838Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3665378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3665505Z outputs = layer_module( 2025-12-04T09:53:14.3666042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3666154Z outputs = self.rel_attn( 2025-12-04T09:53:14.3666793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3666967Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3666983Z 2025-12-04T09:53:14.3667337Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3667744Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3667852Z res = mod(**inputs) 2025-12-04T09:53:14.3668419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3668559Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3669132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3669247Z outputs = layer_module( 2025-12-04T09:53:14.3669806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3669934Z outputs = self.rel_attn( 2025-12-04T09:53:14.3670553Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3670681Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3671315Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3671560Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3671577Z 2025-12-04T09:53:14.3671777Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3672219Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3672329Z res = mod(**inputs) 2025-12-04T09:53:14.3672911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3673056Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3673622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3673752Z outputs = layer_module( 2025-12-04T09:53:14.3674312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3674442Z outputs = self.rel_attn( 2025-12-04T09:53:14.3675002Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3675247Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3675263Z 2025-12-04T09:53:14.3675468Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3675879Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3675996Z res = mod(**inputs) 2025-12-04T09:53:14.3676561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3676710Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3677293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3677414Z outputs = layer_module( 2025-12-04T09:53:14.3677974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3678105Z outputs = self.rel_attn( 2025-12-04T09:53:14.3678670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3678800Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3679500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3679739Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3679759Z 2025-12-04T09:53:14.3679951Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3680352Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3680468Z res = mod(**inputs) 2025-12-04T09:53:14.3681018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3681159Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3681732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3681849Z outputs = layer_module( 2025-12-04T09:53:14.3682389Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3682515Z outputs = self.rel_attn( 2025-12-04T09:53:14.3683112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3683294Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3683309Z 2025-12-04T09:53:14.3683520Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3683924Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3684046Z res = mod(**inputs) 2025-12-04T09:53:14.3684675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3684834Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3685382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3685500Z outputs = layer_module( 2025-12-04T09:53:14.3686078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3686281Z outputs = self.rel_attn( 2025-12-04T09:53:14.3687109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3687312Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3687909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3688149Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3688165Z 2025-12-04T09:53:14.3688355Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3688767Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3688888Z res = mod(**inputs) 2025-12-04T09:53:14.3689451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3689616Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3690177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3690297Z outputs = layer_module( 2025-12-04T09:53:14.3690864Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3690982Z outputs = self.rel_attn( 2025-12-04T09:53:14.3691543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3691709Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3692310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3692523Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3692544Z 2025-12-04T09:53:14.3692732Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3693141Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3693263Z res = mod(**inputs) 2025-12-04T09:53:14.3693827Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3693967Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3694546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3694662Z outputs = layer_module( 2025-12-04T09:53:14.3695233Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3695354Z outputs = self.rel_attn( 2025-12-04T09:53:14.3695909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3696150Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3696787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3696997Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3697013Z 2025-12-04T09:53:14.3697153Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3697368Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3697791Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3697899Z res = mod(**inputs) 2025-12-04T09:53:14.3698572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3698721Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3699276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3699401Z outputs = layer_module( 2025-12-04T09:53:14.3699948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3700365Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3701468Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3701610Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3702189Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3702316Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3702878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3703050Z output = self.activation_function(output) 2025-12-04T09:53:14.3703515Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3703637Z return self.act(input) 2025-12-04T09:53:14.3703662Z 2025-12-04T09:53:14.3703799Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3703987Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3704412Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3704520Z res = mod(**inputs) 2025-12-04T09:53:14.3705086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3705240Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3705803Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3705924Z outputs = layer_module( 2025-12-04T09:53:14.3706496Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3706617Z outputs = self.rel_attn( 2025-12-04T09:53:14.3707259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3707434Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3707450Z 2025-12-04T09:53:14.3707641Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3708064Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3708176Z res = mod(**inputs) 2025-12-04T09:53:14.3708755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3709015Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3709583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3709749Z outputs = layer_module( 2025-12-04T09:53:14.3710311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3710432Z outputs = self.rel_attn( 2025-12-04T09:53:14.3711043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3711221Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3711237Z 2025-12-04T09:53:14.3711437Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3711848Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3711962Z res = mod(**inputs) 2025-12-04T09:53:14.3712540Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3712682Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3713263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3713379Z outputs = layer_module( 2025-12-04T09:53:14.3713940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3714075Z outputs = self.rel_attn( 2025-12-04T09:53:14.3714631Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3714756Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3715364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3715610Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3715626Z 2025-12-04T09:53:14.3715824Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3716239Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3716347Z res = mod(**inputs) 2025-12-04T09:53:14.3716930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3717073Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3717643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3717759Z outputs = layer_module( 2025-12-04T09:53:14.3718316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3718452Z outputs = self.rel_attn( 2025-12-04T09:53:14.3719110Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3719353Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3719368Z 2025-12-04T09:53:14.3719565Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3719965Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3720084Z res = mod(**inputs) 2025-12-04T09:53:14.3720630Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3720771Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3721330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3721502Z outputs = layer_module( 2025-12-04T09:53:14.3722055Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3722197Z outputs = self.rel_attn( 2025-12-04T09:53:14.3722765Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3722900Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3723506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3723749Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3723764Z 2025-12-04T09:53:14.3723957Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3724357Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3724481Z res = mod(**inputs) 2025-12-04T09:53:14.3725026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3725166Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3725720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3725832Z outputs = layer_module( 2025-12-04T09:53:14.3726379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3726501Z outputs = self.rel_attn( 2025-12-04T09:53:14.3727043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3727219Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3727235Z 2025-12-04T09:53:14.3727422Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3727821Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3727941Z res = mod(**inputs) 2025-12-04T09:53:14.3728490Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3728636Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3729184Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3729300Z outputs = layer_module( 2025-12-04T09:53:14.3729859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3729973Z outputs = self.rel_attn( 2025-12-04T09:53:14.3730510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3730646Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3731223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3731451Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3731466Z 2025-12-04T09:53:14.3731652Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3732053Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3732174Z res = mod(**inputs) 2025-12-04T09:53:14.3732722Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3732872Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3733417Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3733585Z outputs = layer_module( 2025-12-04T09:53:14.3734141Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3734255Z outputs = self.rel_attn( 2025-12-04T09:53:14.3734836Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3735000Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3735609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3735820Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3735835Z 2025-12-04T09:53:14.3736021Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3736420Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3736544Z res = mod(**inputs) 2025-12-04T09:53:14.3737089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3737236Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3737785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3737895Z outputs = layer_module( 2025-12-04T09:53:14.3738488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3738608Z outputs = self.rel_attn( 2025-12-04T09:53:14.3739149Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3739309Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3739895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3740109Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3740124Z 2025-12-04T09:53:14.3740261Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3740448Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3740859Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3740968Z res = mod(**inputs) 2025-12-04T09:53:14.3741519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3741673Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3742221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3742344Z outputs = layer_module( 2025-12-04T09:53:14.3742887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3743307Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3743879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3744010Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3744575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3744697Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3745245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3745401Z output = self.activation_function(output) 2025-12-04T09:53:14.3745846Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3746020Z return self.act(input) 2025-12-04T09:53:14.3746044Z 2025-12-04T09:53:14.3746178Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3746364Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3746889Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3746999Z res = mod(**inputs) 2025-12-04T09:53:14.3747761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3747916Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3748478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3748658Z outputs = layer_module( 2025-12-04T09:53:14.3755506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3755637Z outputs = self.rel_attn( 2025-12-04T09:53:14.3756214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3756384Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3756399Z 2025-12-04T09:53:14.3756589Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3757012Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3757122Z res = mod(**inputs) 2025-12-04T09:53:14.3757699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3757844Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3758406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3758540Z outputs = layer_module( 2025-12-04T09:53:14.3759101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3759222Z outputs = self.rel_attn( 2025-12-04T09:53:14.3759793Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3759964Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3759980Z 2025-12-04T09:53:14.3760180Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3760708Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3760813Z res = mod(**inputs) 2025-12-04T09:53:14.3761367Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3761505Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3762066Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3762180Z outputs = layer_module( 2025-12-04T09:53:14.3762723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3762850Z outputs = self.rel_attn( 2025-12-04T09:53:14.3763398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3763519Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3764109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3764343Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3764357Z 2025-12-04T09:53:14.3764550Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3765016Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3765118Z res = mod(**inputs) 2025-12-04T09:53:14.3765707Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3765847Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3766401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3766539Z outputs = layer_module( 2025-12-04T09:53:14.3767083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3767208Z outputs = self.rel_attn( 2025-12-04T09:53:14.3767749Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3767992Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3768016Z 2025-12-04T09:53:14.3768197Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3768602Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3768714Z res = mod(**inputs) 2025-12-04T09:53:14.3769263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3769407Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3769964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3770076Z outputs = layer_module( 2025-12-04T09:53:14.3770617Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3770733Z outputs = self.rel_attn( 2025-12-04T09:53:14.3771274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3771401Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3771979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3772218Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3772242Z 2025-12-04T09:53:14.3772426Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3772829Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3772947Z res = mod(**inputs) 2025-12-04T09:53:14.3773495Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3773636Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3774197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3774312Z outputs = layer_module( 2025-12-04T09:53:14.3774865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3774979Z outputs = self.rel_attn( 2025-12-04T09:53:14.3775526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3775706Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3775721Z 2025-12-04T09:53:14.3775902Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3776301Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3776417Z res = mod(**inputs) 2025-12-04T09:53:14.3776959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3777165Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3777736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3777851Z outputs = layer_module( 2025-12-04T09:53:14.3778402Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3778545Z outputs = self.rel_attn( 2025-12-04T09:53:14.3779086Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3779215Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3779796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3780031Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3780046Z 2025-12-04T09:53:14.3780226Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3780630Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3780747Z res = mod(**inputs) 2025-12-04T09:53:14.3781294Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3781452Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3782003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3782114Z outputs = layer_module( 2025-12-04T09:53:14.3782669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3782785Z outputs = self.rel_attn( 2025-12-04T09:53:14.3783326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3783484Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3784068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3784278Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3784294Z 2025-12-04T09:53:14.3784496Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3784892Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3785010Z res = mod(**inputs) 2025-12-04T09:53:14.3785559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3785711Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3786264Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3786374Z outputs = layer_module( 2025-12-04T09:53:14.3787026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3787311Z outputs = self.rel_attn( 2025-12-04T09:53:14.3787961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3788127Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3788731Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3788940Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3788956Z 2025-12-04T09:53:14.3789096Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3789363Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3789781Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3789893Z res = mod(**inputs) 2025-12-04T09:53:14.3790503Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3790647Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3791238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3791366Z outputs = layer_module( 2025-12-04T09:53:14.3791926Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3792358Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3792943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3793077Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3793654Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3793780Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3794342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3794501Z output = self.activation_function(output) 2025-12-04T09:53:14.3794962Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3795092Z return self.act(input) 2025-12-04T09:53:14.3795107Z 2025-12-04T09:53:14.3795244Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3795435Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3795863Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3795971Z res = mod(**inputs) 2025-12-04T09:53:14.3796538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3796687Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3797247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3797375Z outputs = layer_module( 2025-12-04T09:53:14.3797931Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3798047Z outputs = self.rel_attn( 2025-12-04T09:53:14.3798612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3798781Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3798798Z 2025-12-04T09:53:14.3799095Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3799501Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3799608Z res = mod(**inputs) 2025-12-04T09:53:14.3800164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3800303Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3801391Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3801537Z outputs = layer_module( 2025-12-04T09:53:14.3802106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3802242Z outputs = self.rel_attn( 2025-12-04T09:53:14.3802909Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3803087Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3803103Z 2025-12-04T09:53:14.3803336Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3803750Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3803861Z res = mod(**inputs) 2025-12-04T09:53:14.3804477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3804621Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3805195Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3805308Z outputs = layer_module( 2025-12-04T09:53:14.3805863Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3805996Z outputs = self.rel_attn( 2025-12-04T09:53:14.3806560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3806695Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3807285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3807531Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3807547Z 2025-12-04T09:53:14.3807747Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3808158Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3808269Z res = mod(**inputs) 2025-12-04T09:53:14.3808847Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3808993Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3809567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3809688Z outputs = layer_module( 2025-12-04T09:53:14.3810246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3810379Z outputs = self.rel_attn( 2025-12-04T09:53:14.3810941Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3811265Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3811281Z 2025-12-04T09:53:14.3811471Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3815398Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3815528Z res = mod(**inputs) 2025-12-04T09:53:14.3816155Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3816301Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3816876Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3816992Z outputs = layer_module( 2025-12-04T09:53:14.3817559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3817681Z outputs = self.rel_attn( 2025-12-04T09:53:14.3818242Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3818373Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3819041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3819281Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3819309Z 2025-12-04T09:53:14.3819528Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3819937Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3820055Z res = mod(**inputs) 2025-12-04T09:53:14.3820649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3820793Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3821373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3821494Z outputs = layer_module( 2025-12-04T09:53:14.3822058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3822181Z outputs = self.rel_attn( 2025-12-04T09:53:14.3822744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3822930Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3822946Z 2025-12-04T09:53:14.3823134Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3823663Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3823776Z res = mod(**inputs) 2025-12-04T09:53:14.3824325Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3824469Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3825012Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3825128Z outputs = layer_module( 2025-12-04T09:53:14.3825675Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3825792Z outputs = self.rel_attn( 2025-12-04T09:53:14.3826340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3826458Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3827299Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3827536Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3827552Z 2025-12-04T09:53:14.3827740Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3828150Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3828276Z res = mod(**inputs) 2025-12-04T09:53:14.3828840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3828997Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3829559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3829675Z outputs = layer_module( 2025-12-04T09:53:14.3830248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3830368Z outputs = self.rel_attn( 2025-12-04T09:53:14.3830936Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3831085Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3831682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3831956Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3831972Z 2025-12-04T09:53:14.3832185Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3832596Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3832715Z res = mod(**inputs) 2025-12-04T09:53:14.3833307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3833460Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3834018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3834134Z outputs = layer_module( 2025-12-04T09:53:14.3834704Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3834826Z outputs = self.rel_attn( 2025-12-04T09:53:14.3835385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3835550Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3836150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3836367Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3836383Z 2025-12-04T09:53:14.3836519Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3836705Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3837124Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3837231Z res = mod(**inputs) 2025-12-04T09:53:14.3837813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3837956Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3838627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3838753Z outputs = layer_module( 2025-12-04T09:53:14.3839292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3839705Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3840268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3840394Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3840947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3841073Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3841611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3841772Z output = self.activation_function(output) 2025-12-04T09:53:14.3842216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3842340Z return self.act(input) 2025-12-04T09:53:14.3842357Z 2025-12-04T09:53:14.3842491Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3842675Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3843079Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3843182Z res = mod(**inputs) 2025-12-04T09:53:14.3843727Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3843943Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3844488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3844637Z outputs = layer_module( 2025-12-04T09:53:14.3845176Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3845288Z outputs = self.rel_attn( 2025-12-04T09:53:14.3845866Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3846030Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3846044Z 2025-12-04T09:53:14.3846234Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3846635Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3846743Z res = mod(**inputs) 2025-12-04T09:53:14.3847296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3847433Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3847982Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3848102Z outputs = layer_module( 2025-12-04T09:53:14.3848642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3848763Z outputs = self.rel_attn( 2025-12-04T09:53:14.3849304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3849473Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3849488Z 2025-12-04T09:53:14.3849682Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3850076Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3850178Z res = mod(**inputs) 2025-12-04T09:53:14.3850734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3850870Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3851428Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3851540Z outputs = layer_module( 2025-12-04T09:53:14.3852084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3852209Z outputs = self.rel_attn( 2025-12-04T09:53:14.3852753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3852886Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3853459Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3853695Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3853710Z 2025-12-04T09:53:14.3853901Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3854298Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3854407Z res = mod(**inputs) 2025-12-04T09:53:14.3854960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3855098Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3855653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3855818Z outputs = layer_module( 2025-12-04T09:53:14.3856356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3856482Z outputs = self.rel_attn( 2025-12-04T09:53:14.3857053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3857305Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3857319Z 2025-12-04T09:53:14.3857528Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3857926Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3858043Z res = mod(**inputs) 2025-12-04T09:53:14.3858589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3858730Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3859290Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3859402Z outputs = layer_module( 2025-12-04T09:53:14.3859952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3860069Z outputs = self.rel_attn( 2025-12-04T09:53:14.3860610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3860739Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3861316Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3861561Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3861576Z 2025-12-04T09:53:14.3861766Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3862162Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3862274Z res = mod(**inputs) 2025-12-04T09:53:14.3862826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3862963Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3863520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3863637Z outputs = layer_module( 2025-12-04T09:53:14.3864179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3864293Z outputs = self.rel_attn( 2025-12-04T09:53:14.3864835Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3865014Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3865028Z 2025-12-04T09:53:14.3865212Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3865619Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3865725Z res = mod(**inputs) 2025-12-04T09:53:14.3866268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3866413Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3867059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3867347Z outputs = layer_module( 2025-12-04T09:53:14.3867933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3868120Z outputs = self.rel_attn( 2025-12-04T09:53:14.3868685Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3868808Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3869436Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3869666Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3869682Z 2025-12-04T09:53:14.3869903Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3870326Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3870435Z res = mod(**inputs) 2025-12-04T09:53:14.3871006Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3871164Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3871734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3871853Z outputs = layer_module( 2025-12-04T09:53:14.3872426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3872546Z outputs = self.rel_attn( 2025-12-04T09:53:14.3873122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3873271Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3873940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3874260Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3874277Z 2025-12-04T09:53:14.3874463Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3874877Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3874997Z res = mod(**inputs) 2025-12-04T09:53:14.3875563Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3875717Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3876281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3876395Z outputs = layer_module( 2025-12-04T09:53:14.3876957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3877072Z outputs = self.rel_attn( 2025-12-04T09:53:14.3877635Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3877788Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3878395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3878668Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3878685Z 2025-12-04T09:53:14.3878823Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3879010Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3879428Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3879533Z res = mod(**inputs) 2025-12-04T09:53:14.3880112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3880255Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3880810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3881000Z outputs = layer_module( 2025-12-04T09:53:14.3881559Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3882020Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3882642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3882774Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3883348Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3883472Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3884024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3884184Z output = self.activation_function(output) 2025-12-04T09:53:14.3884641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3884765Z return self.act(input) 2025-12-04T09:53:14.3884782Z 2025-12-04T09:53:14.3884919Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3885107Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3885524Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3885630Z res = mod(**inputs) 2025-12-04T09:53:14.3886292Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3886437Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3886979Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3887100Z outputs = layer_module( 2025-12-04T09:53:14.3887642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3887756Z outputs = self.rel_attn( 2025-12-04T09:53:14.3888298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3888460Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3888474Z 2025-12-04T09:53:14.3888663Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3889058Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3889164Z res = mod(**inputs) 2025-12-04T09:53:14.3889709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3889847Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3890395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3890511Z outputs = layer_module( 2025-12-04T09:53:14.3891050Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3891170Z outputs = self.rel_attn( 2025-12-04T09:53:14.3891709Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3891878Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3891893Z 2025-12-04T09:53:14.3892081Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3892474Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3892588Z res = mod(**inputs) 2025-12-04T09:53:14.3893190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3893330Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3893960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3894077Z outputs = layer_module( 2025-12-04T09:53:14.3894620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3894772Z outputs = self.rel_attn( 2025-12-04T09:53:14.3895310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3895433Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3896003Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3896242Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3896257Z 2025-12-04T09:53:14.3896448Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3896848Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3896963Z res = mod(**inputs) 2025-12-04T09:53:14.3897507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3897645Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3898203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3898315Z outputs = layer_module( 2025-12-04T09:53:14.3898856Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3898981Z outputs = self.rel_attn( 2025-12-04T09:53:14.3899528Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3899775Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3899790Z 2025-12-04T09:53:14.3899974Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3900371Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3900485Z res = mod(**inputs) 2025-12-04T09:53:14.3901571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3901719Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3902298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3902417Z outputs = layer_module( 2025-12-04T09:53:14.3902990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3903110Z outputs = self.rel_attn( 2025-12-04T09:53:14.3903670Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3903808Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3904409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3904659Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3904675Z 2025-12-04T09:53:14.3904868Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3905279Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3905400Z res = mod(**inputs) 2025-12-04T09:53:14.3906088Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3906232Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3906948Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3907067Z outputs = layer_module( 2025-12-04T09:53:14.3907633Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3907796Z outputs = self.rel_attn( 2025-12-04T09:53:14.3908357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3908538Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3908554Z 2025-12-04T09:53:14.3908745Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3909165Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3909278Z res = mod(**inputs) 2025-12-04T09:53:14.3909841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3909996Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3910557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3910673Z outputs = layer_module( 2025-12-04T09:53:14.3911244Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3911367Z outputs = self.rel_attn( 2025-12-04T09:53:14.3911938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3912064Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3912660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3912896Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3912912Z 2025-12-04T09:53:14.3913105Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3913529Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3913638Z res = mod(**inputs) 2025-12-04T09:53:14.3914205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3914359Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3914924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3915043Z outputs = layer_module( 2025-12-04T09:53:14.3915621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3915741Z outputs = self.rel_attn( 2025-12-04T09:53:14.3916312Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3916463Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3917071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3917494Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3917510Z 2025-12-04T09:53:14.3917755Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3918267Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3918414Z res = mod(**inputs) 2025-12-04T09:53:14.3919161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3919378Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3920043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3920263Z outputs = layer_module( 2025-12-04T09:53:14.3920840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3921066Z outputs = self.rel_attn( 2025-12-04T09:53:14.3921647Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3921821Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3922538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3922777Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3922792Z 2025-12-04T09:53:14.3923009Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3923228Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3923677Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3923877Z res = mod(**inputs) 2025-12-04T09:53:14.3924507Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3934346Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3935007Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3935137Z outputs = layer_module( 2025-12-04T09:53:14.3935691Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3936220Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3937085Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3937275Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3937848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3937979Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3938554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3938705Z output = self.activation_function(output) 2025-12-04T09:53:14.3939163Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3939298Z return self.act(input) 2025-12-04T09:53:14.3939321Z 2025-12-04T09:53:14.3939463Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3939655Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3940077Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3940184Z res = mod(**inputs) 2025-12-04T09:53:14.3940754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3940901Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3941462Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3941585Z outputs = layer_module( 2025-12-04T09:53:14.3942143Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3942260Z outputs = self.rel_attn( 2025-12-04T09:53:14.3942945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3943115Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3943131Z 2025-12-04T09:53:14.3943368Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3943780Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3943890Z res = mod(**inputs) 2025-12-04T09:53:14.3944498Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3944644Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3945215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3945335Z outputs = layer_module( 2025-12-04T09:53:14.3945895Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3946024Z outputs = self.rel_attn( 2025-12-04T09:53:14.3946587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3946848Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3946877Z 2025-12-04T09:53:14.3947069Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3947489Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3947607Z res = mod(**inputs) 2025-12-04T09:53:14.3948173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3948317Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3948889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3949013Z outputs = layer_module( 2025-12-04T09:53:14.3949580Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3949703Z outputs = self.rel_attn( 2025-12-04T09:53:14.3950259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3950391Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3950987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3951238Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3951254Z 2025-12-04T09:53:14.3951451Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3951861Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3951985Z res = mod(**inputs) 2025-12-04T09:53:14.3952549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3952695Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3953266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3953383Z outputs = layer_module( 2025-12-04T09:53:14.3953951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3954071Z outputs = self.rel_attn( 2025-12-04T09:53:14.3954626Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3954883Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3954961Z 2025-12-04T09:53:14.3955153Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3955565Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3955683Z res = mod(**inputs) 2025-12-04T09:53:14.3956276Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3956432Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3957027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3957147Z outputs = layer_module( 2025-12-04T09:53:14.3957716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3957835Z outputs = self.rel_attn( 2025-12-04T09:53:14.3958497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3958631Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3959204Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.3959450Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.3959465Z 2025-12-04T09:53:14.3959647Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3960043Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3960156Z res = mod(**inputs) 2025-12-04T09:53:14.3960696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3960842Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3961385Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3961497Z outputs = layer_module( 2025-12-04T09:53:14.3962044Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3962161Z outputs = self.rel_attn( 2025-12-04T09:53:14.3962695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.3962871Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.3962888Z 2025-12-04T09:53:14.3963069Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3963482Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3963588Z res = mod(**inputs) 2025-12-04T09:53:14.3964132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3964282Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3964824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3964942Z outputs = layer_module( 2025-12-04T09:53:14.3965484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3965600Z outputs = self.rel_attn( 2025-12-04T09:53:14.3966147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3966265Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3966842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.3967064Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.3967149Z 2025-12-04T09:53:14.3967336Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3967742Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3967845Z res = mod(**inputs) 2025-12-04T09:53:14.3968422Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3968566Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3969147Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3969263Z outputs = layer_module( 2025-12-04T09:53:14.3969798Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3969913Z outputs = self.rel_attn( 2025-12-04T09:53:14.3970464Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3970615Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3971198Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3971405Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3971420Z 2025-12-04T09:53:14.3971599Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3972009Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3972114Z res = mod(**inputs) 2025-12-04T09:53:14.3972666Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3972809Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3973356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3973485Z outputs = layer_module( 2025-12-04T09:53:14.3974023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3974142Z outputs = self.rel_attn( 2025-12-04T09:53:14.3974694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.3974840Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.3975430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.3975635Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.3975650Z 2025-12-04T09:53:14.3975789Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3975986Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3976387Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3976493Z res = mod(**inputs) 2025-12-04T09:53:14.3977054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3977196Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3977739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3977866Z outputs = layer_module( 2025-12-04T09:53:14.3978403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.3978832Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.3979390Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.3979573Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.3980136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.3980286Z output_x = self.ff(output_x) 2025-12-04T09:53:14.3980844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.3980992Z output = self.activation_function(output) 2025-12-04T09:53:14.3981465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.3981590Z return self.act(input) 2025-12-04T09:53:14.3981605Z 2025-12-04T09:53:14.3981738Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.3981919Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3982324Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3982432Z res = mod(**inputs) 2025-12-04T09:53:14.3982987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3983129Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3983671Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3983793Z outputs = layer_module( 2025-12-04T09:53:14.3984340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3984458Z outputs = self.rel_attn( 2025-12-04T09:53:14.3984994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.3985159Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.3985175Z 2025-12-04T09:53:14.3985366Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3985764Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3985867Z res = mod(**inputs) 2025-12-04T09:53:14.3986430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3986572Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3987401Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3987521Z outputs = layer_module( 2025-12-04T09:53:14.3988079Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3988206Z outputs = self.rel_attn( 2025-12-04T09:53:14.3988767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.3988944Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.3988970Z 2025-12-04T09:53:14.3989157Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3989569Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3989686Z res = mod(**inputs) 2025-12-04T09:53:14.3990248Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3990390Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3990957Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3991070Z outputs = layer_module( 2025-12-04T09:53:14.3991638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3991823Z outputs = self.rel_attn( 2025-12-04T09:53:14.3992384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.3992514Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.3993136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.3993384Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.3993412Z 2025-12-04T09:53:14.3993626Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3994039Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3994161Z res = mod(**inputs) 2025-12-04T09:53:14.3994725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3994872Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.3995442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.3995555Z outputs = layer_module( 2025-12-04T09:53:14.3996120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.3996237Z outputs = self.rel_attn( 2025-12-04T09:53:14.3996796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.3997049Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.3997065Z 2025-12-04T09:53:14.3997249Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.3997655Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.3997770Z res = mod(**inputs) 2025-12-04T09:53:14.3998337Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.3998506Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4006058Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4006187Z outputs = layer_module( 2025-12-04T09:53:14.4006760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4006877Z outputs = self.rel_attn( 2025-12-04T09:53:14.4007442Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4007565Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4008161Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.4008416Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.4008433Z 2025-12-04T09:53:14.4008622Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4009039Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4009157Z res = mod(**inputs) 2025-12-04T09:53:14.4009720Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4009872Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4010433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4010549Z outputs = layer_module( 2025-12-04T09:53:14.4011113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4011950Z outputs = self.rel_attn( 2025-12-04T09:53:14.4012521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.4012698Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.4012756Z 2025-12-04T09:53:14.4013058Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4013465Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4013626Z res = mod(**inputs) 2025-12-04T09:53:14.4014174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4014325Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4014871Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4014998Z outputs = layer_module( 2025-12-04T09:53:14.4015539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4015659Z outputs = self.rel_attn( 2025-12-04T09:53:14.4016219Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4016343Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4016927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.4017155Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.4017170Z 2025-12-04T09:53:14.4017354Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4017767Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4017872Z res = mod(**inputs) 2025-12-04T09:53:14.4018425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4018574Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4019117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4019241Z outputs = layer_module( 2025-12-04T09:53:14.4019777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4019897Z outputs = self.rel_attn( 2025-12-04T09:53:14.4020453Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4020600Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4021183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4021389Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4021405Z 2025-12-04T09:53:14.4021585Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4021992Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4022098Z res = mod(**inputs) 2025-12-04T09:53:14.4022643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4022794Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4023342Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4023466Z outputs = layer_module( 2025-12-04T09:53:14.4024005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4024175Z outputs = self.rel_attn( 2025-12-04T09:53:14.4024724Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4024872Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4025485Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4025697Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4025712Z 2025-12-04T09:53:14.4025873Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4026061Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4026457Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4026571Z res = mod(**inputs) 2025-12-04T09:53:14.4027381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4027545Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4028106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4028223Z outputs = layer_module( 2025-12-04T09:53:14.4028794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.4029224Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.4029804Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.4029934Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.4030493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.4030634Z output_x = self.ff(output_x) 2025-12-04T09:53:14.4031190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.4031337Z output = self.activation_function(output) 2025-12-04T09:53:14.4031801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.4031917Z return self.act(input) 2025-12-04T09:53:14.4031933Z 2025-12-04T09:53:14.4032074Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4032264Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4032671Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4032783Z res = mod(**inputs) 2025-12-04T09:53:14.4033349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4033488Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4034064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4034180Z outputs = layer_module( 2025-12-04T09:53:14.4034745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4034862Z outputs = self.rel_attn( 2025-12-04T09:53:14.4035419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.4035587Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.4035602Z 2025-12-04T09:53:14.4035789Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4036204Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4036313Z res = mod(**inputs) 2025-12-04T09:53:14.4036945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4037090Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4037682Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4037797Z outputs = layer_module( 2025-12-04T09:53:14.4038358Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4038504Z outputs = self.rel_attn( 2025-12-04T09:53:14.4039183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.4039347Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.4039363Z 2025-12-04T09:53:14.4039541Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4039952Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4040054Z res = mod(**inputs) 2025-12-04T09:53:14.4040605Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4040745Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4041283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4041404Z outputs = layer_module( 2025-12-04T09:53:14.4041940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4042053Z outputs = self.rel_attn( 2025-12-04T09:53:14.4042598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4042716Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4043301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.4043536Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.4043551Z 2025-12-04T09:53:14.4043732Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4044134Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4044238Z res = mod(**inputs) 2025-12-04T09:53:14.4044795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4044931Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4045472Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4045590Z outputs = layer_module( 2025-12-04T09:53:14.4046132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4046245Z outputs = self.rel_attn( 2025-12-04T09:53:14.4046792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.4047026Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.4047041Z 2025-12-04T09:53:14.4047231Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4047628Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4047730Z res = mod(**inputs) 2025-12-04T09:53:14.4048280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4048414Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4048966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4049133Z outputs = layer_module( 2025-12-04T09:53:14.4049701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4049825Z outputs = self.rel_attn( 2025-12-04T09:53:14.4050362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4050504Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4051092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.4051325Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.4051341Z 2025-12-04T09:53:14.4051529Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4051933Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4052037Z res = mod(**inputs) 2025-12-04T09:53:14.4052600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4052739Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4053293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4053411Z outputs = layer_module( 2025-12-04T09:53:14.4053947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4054073Z outputs = self.rel_attn( 2025-12-04T09:53:14.4054612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.4054776Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.4054796Z 2025-12-04T09:53:14.4054985Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4055380Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4055497Z res = mod(**inputs) 2025-12-04T09:53:14.4056042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4056175Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4056732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4056842Z outputs = layer_module( 2025-12-04T09:53:14.4057378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4057505Z outputs = self.rel_attn( 2025-12-04T09:53:14.4058041Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4058175Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4058747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.4058957Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.4058971Z 2025-12-04T09:53:14.4059155Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4059549Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4059658Z res = mod(**inputs) 2025-12-04T09:53:14.4060202Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4060335Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4060885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4061168Z outputs = layer_module( 2025-12-04T09:53:14.4063784Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4063979Z outputs = self.rel_attn( 2025-12-04T09:53:14.4064538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4064718Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4065322Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4065519Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4065536Z 2025-12-04T09:53:14.4065726Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4066202Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4066320Z res = mod(**inputs) 2025-12-04T09:53:14.4066988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4067130Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4067697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4067808Z outputs = layer_module( 2025-12-04T09:53:14.4068363Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4068491Z outputs = self.rel_attn( 2025-12-04T09:53:14.4069043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4069200Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4069801Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4070000Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4070016Z 2025-12-04T09:53:14.4070161Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4070345Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4070761Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4070870Z res = mod(**inputs) 2025-12-04T09:53:14.4071432Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4071581Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4072136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4072255Z outputs = layer_module( 2025-12-04T09:53:14.4072820Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.4073248Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.4073831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.4073961Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.4074520Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.4074649Z output_x = self.ff(output_x) 2025-12-04T09:53:14.4075205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.4075360Z output = self.activation_function(output) 2025-12-04T09:53:14.4075875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.4075989Z return self.act(input) 2025-12-04T09:53:14.4076004Z 2025-12-04T09:53:14.4076147Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4076360Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4076770Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4076884Z res = mod(**inputs) 2025-12-04T09:53:14.4077471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4077619Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4078177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4078290Z outputs = layer_module( 2025-12-04T09:53:14.4078949Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4079061Z outputs = self.rel_attn( 2025-12-04T09:53:14.4079602Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.4079771Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.4079785Z 2025-12-04T09:53:14.4079965Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4080372Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4080473Z res = mod(**inputs) 2025-12-04T09:53:14.4081010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4081154Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4081696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4081817Z outputs = layer_module( 2025-12-04T09:53:14.4082354Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4082467Z outputs = self.rel_attn( 2025-12-04T09:53:14.4083016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.4083184Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.4083199Z 2025-12-04T09:53:14.4083375Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4083777Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4083879Z res = mod(**inputs) 2025-12-04T09:53:14.4084431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4084572Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4085115Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4085236Z outputs = layer_module( 2025-12-04T09:53:14.4085782Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4085896Z outputs = self.rel_attn( 2025-12-04T09:53:14.4086447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4086564Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4087153Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.4087387Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.4087456Z 2025-12-04T09:53:14.4087640Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4088049Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4088152Z res = mod(**inputs) 2025-12-04T09:53:14.4088735Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4088873Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4089445Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4089568Z outputs = layer_module( 2025-12-04T09:53:14.4090103Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4090217Z outputs = self.rel_attn( 2025-12-04T09:53:14.4090768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.4091009Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.4091023Z 2025-12-04T09:53:14.4091213Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4091610Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4091710Z res = mod(**inputs) 2025-12-04T09:53:14.4092263Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4092398Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4092945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4093054Z outputs = layer_module( 2025-12-04T09:53:14.4093586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4093711Z outputs = self.rel_attn( 2025-12-04T09:53:14.4094245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4094361Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4094943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.4095176Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.4095190Z 2025-12-04T09:53:14.4095381Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4095773Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4095873Z res = mod(**inputs) 2025-12-04T09:53:14.4096426Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4096564Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4097112Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4097221Z outputs = layer_module( 2025-12-04T09:53:14.4097760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4097880Z outputs = self.rel_attn( 2025-12-04T09:53:14.4098419Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.4098586Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.4098611Z 2025-12-04T09:53:14.4098790Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4099181Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4099356Z res = mod(**inputs) 2025-12-04T09:53:14.4099903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4100042Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4100620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4101078Z outputs = layer_module( 2025-12-04T09:53:14.4101905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4102031Z outputs = self.rel_attn( 2025-12-04T09:53:14.4102590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4102717Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4103305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.4103529Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.4103546Z 2025-12-04T09:53:14.4103740Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4104150Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4104267Z res = mod(**inputs) 2025-12-04T09:53:14.4104833Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4104976Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4105542Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4105655Z outputs = layer_module( 2025-12-04T09:53:14.4106209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4106339Z outputs = self.rel_attn( 2025-12-04T09:53:14.4106977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4107143Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4107746Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4107949Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4107967Z 2025-12-04T09:53:14.4108165Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4108575Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4108695Z res = mod(**inputs) 2025-12-04T09:53:14.4109259Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4109406Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4109981Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4110097Z outputs = layer_module( 2025-12-04T09:53:14.4110652Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4110776Z outputs = self.rel_attn( 2025-12-04T09:53:14.4111330Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4111489Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4112090Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4112292Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4112405Z 2025-12-04T09:53:14.4112550Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4112734Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4113151Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4113294Z res = mod(**inputs) 2025-12-04T09:53:14.4113858Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4114010Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4114597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4114711Z outputs = layer_module( 2025-12-04T09:53:14.4115273Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.4115696Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.4116277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.4116405Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.4116965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.4117091Z output_x = self.ff(output_x) 2025-12-04T09:53:14.4117645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.4117798Z output = self.activation_function(output) 2025-12-04T09:53:14.4118253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.4118368Z return self.act(input) 2025-12-04T09:53:14.4118383Z 2025-12-04T09:53:14.4118530Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4118720Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4119232Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4119345Z res = mod(**inputs) 2025-12-04T09:53:14.4119890Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4120038Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4120587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4120696Z outputs = layer_module( 2025-12-04T09:53:14.4121247Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4121359Z outputs = self.rel_attn( 2025-12-04T09:53:14.4121897Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.4122067Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.4122082Z 2025-12-04T09:53:14.4122259Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4122661Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4122763Z res = mod(**inputs) 2025-12-04T09:53:14.4123308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4123453Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4124336Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4124460Z outputs = layer_module( 2025-12-04T09:53:14.4125017Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4125230Z outputs = self.rel_attn( 2025-12-04T09:53:14.4125792Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.4125991Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.4126006Z 2025-12-04T09:53:14.4126193Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4126604Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4126737Z res = mod(**inputs) 2025-12-04T09:53:14.4127307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4127445Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4128001Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4128126Z outputs = layer_module( 2025-12-04T09:53:14.4128752Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4128872Z outputs = self.rel_attn( 2025-12-04T09:53:14.4129431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4129551Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4130152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.4130393Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.4130409Z 2025-12-04T09:53:14.4130591Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4131006Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4131109Z res = mod(**inputs) 2025-12-04T09:53:14.4131683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4131823Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4132382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4132502Z outputs = layer_module( 2025-12-04T09:53:14.4133056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4133180Z outputs = self.rel_attn( 2025-12-04T09:53:14.4133734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.4133976Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.4133991Z 2025-12-04T09:53:14.4134185Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4134597Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4134702Z res = mod(**inputs) 2025-12-04T09:53:14.4135274Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4135417Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4136113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4136229Z outputs = layer_module( 2025-12-04T09:53:14.4136767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4136892Z outputs = self.rel_attn( 2025-12-04T09:53:14.4137429Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4137607Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4138191Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.4138481Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.4138496Z 2025-12-04T09:53:14.4138686Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4139080Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4139213Z res = mod(**inputs) 2025-12-04T09:53:14.4139764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4139900Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4140454Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4140572Z outputs = layer_module( 2025-12-04T09:53:14.4141108Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4141237Z outputs = self.rel_attn( 2025-12-04T09:53:14.4141779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.4141942Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.4141967Z 2025-12-04T09:53:14.4142149Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4142543Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4142653Z res = mod(**inputs) 2025-12-04T09:53:14.4143200Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4143338Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4143893Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4144002Z outputs = layer_module( 2025-12-04T09:53:14.4144548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4144662Z outputs = self.rel_attn( 2025-12-04T09:53:14.4145196Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4145319Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4145889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.4146103Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.4146127Z 2025-12-04T09:53:14.4146304Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4146796Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4146919Z res = mod(**inputs) 2025-12-04T09:53:14.4147634Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4147775Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4148364Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4148479Z outputs = layer_module( 2025-12-04T09:53:14.4149042Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4149159Z outputs = self.rel_attn( 2025-12-04T09:53:14.4149713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4149871Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4150532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4150730Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4150784Z 2025-12-04T09:53:14.4150973Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4151379Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4151494Z res = mod(**inputs) 2025-12-04T09:53:14.4152082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4152223Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4152794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4152906Z outputs = layer_module( 2025-12-04T09:53:14.4153474Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4153589Z outputs = self.rel_attn( 2025-12-04T09:53:14.4154144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4154301Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4154901Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4155100Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4155115Z 2025-12-04T09:53:14.4155263Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4155446Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4155862Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4155973Z res = mod(**inputs) 2025-12-04T09:53:14.4156535Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4156691Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4157253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4157367Z outputs = layer_module( 2025-12-04T09:53:14.4157946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.4158371Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.4158951Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.4159080Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.4159754Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.4159876Z output_x = self.ff(output_x) 2025-12-04T09:53:14.4160414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.4160562Z output = self.activation_function(output) 2025-12-04T09:53:14.4161005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.4161115Z return self.act(input) 2025-12-04T09:53:14.4161130Z 2025-12-04T09:53:14.4161268Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4161449Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4161840Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4161951Z res = mod(**inputs) 2025-12-04T09:53:14.4162567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4162710Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4163283Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4163398Z outputs = layer_module( 2025-12-04T09:53:14.4163947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4164090Z outputs = self.rel_attn( 2025-12-04T09:53:14.4164643Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.4164806Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.4164821Z 2025-12-04T09:53:14.4165003Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4165412Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4165514Z res = mod(**inputs) 2025-12-04T09:53:14.4166056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4166204Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4166747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4166870Z outputs = layer_module( 2025-12-04T09:53:14.4167408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4167521Z outputs = self.rel_attn( 2025-12-04T09:53:14.4168065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.4168232Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.4168251Z 2025-12-04T09:53:14.4168442Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4168844Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4168953Z res = mod(**inputs) 2025-12-04T09:53:14.4169517Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4169659Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4170208Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4170337Z outputs = layer_module( 2025-12-04T09:53:14.4170875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4170997Z outputs = self.rel_attn( 2025-12-04T09:53:14.4171538Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4171658Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4172243Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.4172478Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.4172492Z 2025-12-04T09:53:14.4172676Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4173077Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4173183Z res = mod(**inputs) 2025-12-04T09:53:14.4173736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4173872Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4174413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4174589Z outputs = layer_module( 2025-12-04T09:53:14.4175154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4175275Z outputs = self.rel_attn( 2025-12-04T09:53:14.4175814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.4176077Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.4176093Z 2025-12-04T09:53:14.4176289Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4176687Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4176796Z res = mod(**inputs) 2025-12-04T09:53:14.4177351Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4177496Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4178051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4178164Z outputs = layer_module( 2025-12-04T09:53:14.4178701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4178820Z outputs = self.rel_attn( 2025-12-04T09:53:14.4179362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4179489Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4180060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.4180289Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.4180308Z 2025-12-04T09:53:14.4180494Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4180887Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4180992Z res = mod(**inputs) 2025-12-04T09:53:14.4181549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4181682Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4182234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4182344Z outputs = layer_module( 2025-12-04T09:53:14.4182880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4183003Z outputs = self.rel_attn( 2025-12-04T09:53:14.4183545Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.4183722Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.4183737Z 2025-12-04T09:53:14.4183920Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4184317Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4184430Z res = mod(**inputs) 2025-12-04T09:53:14.4184977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4185112Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4185659Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4185771Z outputs = layer_module( 2025-12-04T09:53:14.4186392Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4186887Z outputs = self.rel_attn( 2025-12-04T09:53:14.4187451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4187620Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4188214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.4188465Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.4188491Z 2025-12-04T09:53:14.4188680Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4189087Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4189204Z res = mod(**inputs) 2025-12-04T09:53:14.4189764Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4189911Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4190484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4190600Z outputs = layer_module( 2025-12-04T09:53:14.4191164Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4191284Z outputs = self.rel_attn( 2025-12-04T09:53:14.4191844Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4192007Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4192606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4192805Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4192836Z 2025-12-04T09:53:14.4193020Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4193428Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4193539Z res = mod(**inputs) 2025-12-04T09:53:14.4194106Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4194253Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4194828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4194944Z outputs = layer_module( 2025-12-04T09:53:14.4195508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4195625Z outputs = self.rel_attn( 2025-12-04T09:53:14.4196181Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4196342Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4196946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4197146Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4197169Z 2025-12-04T09:53:14.4197305Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4197490Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4197902Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4198008Z res = mod(**inputs) 2025-12-04T09:53:14.4198575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4198718Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4199435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4199554Z outputs = layer_module( 2025-12-04T09:53:14.4200122Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.4200537Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.4201681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.4201821Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.4202386Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.4202515Z output_x = self.ff(output_x) 2025-12-04T09:53:14.4203075Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.4203239Z output = self.activation_function(output) 2025-12-04T09:53:14.4203696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.4203819Z return self.act(input) 2025-12-04T09:53:14.4203835Z 2025-12-04T09:53:14.4203980Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4204169Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4204595Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4204709Z res = mod(**inputs) 2025-12-04T09:53:14.4205267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4205421Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4205978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4206102Z outputs = layer_module( 2025-12-04T09:53:14.4206664Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4206788Z outputs = self.rel_attn( 2025-12-04T09:53:14.4207356Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.4207523Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.4207538Z 2025-12-04T09:53:14.4207725Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4208146Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4208254Z res = mod(**inputs) 2025-12-04T09:53:14.4208813Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4208967Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4209526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4209656Z outputs = layer_module( 2025-12-04T09:53:14.4210216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4210331Z outputs = self.rel_attn( 2025-12-04T09:53:14.4210907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.4211079Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.4211094Z 2025-12-04T09:53:14.4211289Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4211697Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4211902Z res = mod(**inputs) 2025-12-04T09:53:14.4212477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4212615Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4213306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4213427Z outputs = layer_module( 2025-12-04T09:53:14.4214004Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4214127Z outputs = self.rel_attn( 2025-12-04T09:53:14.4214667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4214783Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4215362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.4215601Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.4215616Z 2025-12-04T09:53:14.4215801Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4216198Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4216302Z res = mod(**inputs) 2025-12-04T09:53:14.4216853Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4216992Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4217539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4217656Z outputs = layer_module( 2025-12-04T09:53:14.4218194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4218320Z outputs = self.rel_attn( 2025-12-04T09:53:14.4218860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.4219101Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.4219116Z 2025-12-04T09:53:14.4219304Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4219701Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4219810Z res = mod(**inputs) 2025-12-04T09:53:14.4220357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4220493Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4221049Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4221166Z outputs = layer_module( 2025-12-04T09:53:14.4221706Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4221826Z outputs = self.rel_attn( 2025-12-04T09:53:14.4222371Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4222499Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4223078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.4223310Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.4223325Z 2025-12-04T09:53:14.4223513Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4223909Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4224068Z res = mod(**inputs) 2025-12-04T09:53:14.4224621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4224756Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4225335Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4225447Z outputs = layer_module( 2025-12-04T09:53:14.4226016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4226142Z outputs = self.rel_attn( 2025-12-04T09:53:14.4226761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.4226958Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.4226973Z 2025-12-04T09:53:14.4227326Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4227745Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4227868Z res = mod(**inputs) 2025-12-04T09:53:14.4228431Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4228573Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4229145Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4229257Z outputs = layer_module( 2025-12-04T09:53:14.4229824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4229941Z outputs = self.rel_attn( 2025-12-04T09:53:14.4230501Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4230638Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4231228Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.4231458Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.4231473Z 2025-12-04T09:53:14.4231660Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4232066Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4232179Z res = mod(**inputs) 2025-12-04T09:53:14.4232741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4232880Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4233446Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4233566Z outputs = layer_module( 2025-12-04T09:53:14.4234132Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4234250Z outputs = self.rel_attn( 2025-12-04T09:53:14.4234807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4234963Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4235571Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4235783Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4235798Z 2025-12-04T09:53:14.4235984Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4236396Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4236570Z res = mod(**inputs) 2025-12-04T09:53:14.4237134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4237273Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4237878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4237994Z outputs = layer_module( 2025-12-04T09:53:14.4238589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4238714Z outputs = self.rel_attn( 2025-12-04T09:53:14.4239376Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4239533Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4240118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4240319Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4240345Z 2025-12-04T09:53:14.4240478Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4240659Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4241060Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4241164Z res = mod(**inputs) 2025-12-04T09:53:14.4241712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4241855Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4242398Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4242517Z outputs = layer_module( 2025-12-04T09:53:14.4243054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.4243470Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.4244032Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.4244157Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.4244701Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.4244820Z output_x = self.ff(output_x) 2025-12-04T09:53:14.4245355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.4245503Z output = self.activation_function(output) 2025-12-04T09:53:14.4245943Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.4246057Z return self.act(input) 2025-12-04T09:53:14.4246072Z 2025-12-04T09:53:14.4246209Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4246390Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4246799Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4246903Z res = mod(**inputs) 2025-12-04T09:53:14.4247451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4247599Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4248140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4248254Z outputs = layer_module( 2025-12-04T09:53:14.4248874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4254197Z outputs = self.rel_attn( 2025-12-04T09:53:14.4254772Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.4254988Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.4255004Z 2025-12-04T09:53:14.4255195Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4255612Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4255745Z res = mod(**inputs) 2025-12-04T09:53:14.4256310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4256460Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4257020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4257153Z outputs = layer_module( 2025-12-04T09:53:14.4257710Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4257826Z outputs = self.rel_attn( 2025-12-04T09:53:14.4258395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.4258567Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.4258583Z 2025-12-04T09:53:14.4258783Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4259195Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4259301Z res = mod(**inputs) 2025-12-04T09:53:14.4259874Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4260017Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4260583Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4260708Z outputs = layer_module( 2025-12-04T09:53:14.4261266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4261391Z outputs = self.rel_attn( 2025-12-04T09:53:14.4261947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4262072Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4262673Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.4262918Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.4262934Z 2025-12-04T09:53:14.4263130Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4263544Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4263653Z res = mod(**inputs) 2025-12-04T09:53:14.4264222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4264364Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4265039Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4265163Z outputs = layer_module( 2025-12-04T09:53:14.4265702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4265821Z outputs = self.rel_attn( 2025-12-04T09:53:14.4266360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.4266656Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.4266755Z 2025-12-04T09:53:14.4266969Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4267621Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4267739Z res = mod(**inputs) 2025-12-04T09:53:14.4268306Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4268479Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4269045Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4269159Z outputs = layer_module( 2025-12-04T09:53:14.4269717Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4269842Z outputs = self.rel_attn( 2025-12-04T09:53:14.4270413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4270547Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4271146Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.4271390Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.4271406Z 2025-12-04T09:53:14.4271609Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4272022Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4272135Z res = mod(**inputs) 2025-12-04T09:53:14.4272700Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4272843Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4273416Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4273528Z outputs = layer_module( 2025-12-04T09:53:14.4274089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4274217Z outputs = self.rel_attn( 2025-12-04T09:53:14.4274779Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.4274962Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.4274977Z 2025-12-04T09:53:14.4275163Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4275574Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4275688Z res = mod(**inputs) 2025-12-04T09:53:14.4276251Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4276395Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4276966Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4277083Z outputs = layer_module( 2025-12-04T09:53:14.4277645Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4277766Z outputs = self.rel_attn( 2025-12-04T09:53:14.4278433Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4278559Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4279133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.4279351Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.4279420Z 2025-12-04T09:53:14.4279601Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4279998Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4280138Z res = mod(**inputs) 2025-12-04T09:53:14.4280687Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4280828Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4281413Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4281528Z outputs = layer_module( 2025-12-04T09:53:14.4282080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4282197Z outputs = self.rel_attn( 2025-12-04T09:53:14.4282741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4282893Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4283480Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4283684Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4283699Z 2025-12-04T09:53:14.4283879Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4284281Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4284398Z res = mod(**inputs) 2025-12-04T09:53:14.4284945Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4285083Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4285636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4285747Z outputs = layer_module( 2025-12-04T09:53:14.4286298Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4286412Z outputs = self.rel_attn( 2025-12-04T09:53:14.4286952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4287100Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4287677Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4287877Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4287892Z 2025-12-04T09:53:14.4288021Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4288209Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4288611Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4288715Z res = mod(**inputs) 2025-12-04T09:53:14.4289268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4289414Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4289961Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4290080Z outputs = layer_module( 2025-12-04T09:53:14.4290618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.4291033Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.4291597Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.4291780Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.4292357Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.4292477Z output_x = self.ff(output_x) 2025-12-04T09:53:14.4293019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.4293197Z output = self.activation_function(output) 2025-12-04T09:53:14.4293641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.4293751Z return self.act(input) 2025-12-04T09:53:14.4293766Z 2025-12-04T09:53:14.4293905Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4294084Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4294489Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4294591Z res = mod(**inputs) 2025-12-04T09:53:14.4295140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4295289Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4295842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4295957Z outputs = layer_module( 2025-12-04T09:53:14.4296510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4296628Z outputs = self.rel_attn( 2025-12-04T09:53:14.4297179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.4297344Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.4297364Z 2025-12-04T09:53:14.4297546Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4297952Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4298061Z res = mod(**inputs) 2025-12-04T09:53:14.4298620Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4298756Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4299300Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4299417Z outputs = layer_module( 2025-12-04T09:53:14.4299959Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4300076Z outputs = self.rel_attn( 2025-12-04T09:53:14.4300625Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.4301326Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.4301345Z 2025-12-04T09:53:14.4301556Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4301974Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4302086Z res = mod(**inputs) 2025-12-04T09:53:14.4302665Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4302807Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4303373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4303487Z outputs = layer_module( 2025-12-04T09:53:14.4304048Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4304298Z outputs = self.rel_attn( 2025-12-04T09:53:14.4304859Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4305019Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4305623Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.4305909Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.4305926Z 2025-12-04T09:53:14.4306121Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4306531Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4306637Z res = mod(**inputs) 2025-12-04T09:53:14.4307287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4307436Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4307996Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4308126Z outputs = layer_module( 2025-12-04T09:53:14.4308679Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4308805Z outputs = self.rel_attn( 2025-12-04T09:53:14.4309362Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.4309604Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.4309620Z 2025-12-04T09:53:14.4309814Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4310223Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4310340Z res = mod(**inputs) 2025-12-04T09:53:14.4310906Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4311112Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4316397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4316516Z outputs = layer_module( 2025-12-04T09:53:14.4317076Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4317204Z outputs = self.rel_attn( 2025-12-04T09:53:14.4317762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4317893Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4318488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.4318733Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.4318749Z 2025-12-04T09:53:14.4318951Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4319367Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4319486Z res = mod(**inputs) 2025-12-04T09:53:14.4320051Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4320195Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4320767Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4320883Z outputs = layer_module( 2025-12-04T09:53:14.4321443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4321644Z outputs = self.rel_attn( 2025-12-04T09:53:14.4322203Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.4322411Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.4322427Z 2025-12-04T09:53:14.4322617Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4323054Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4323167Z res = mod(**inputs) 2025-12-04T09:53:14.4323838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4323982Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4324527Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4324644Z outputs = layer_module( 2025-12-04T09:53:14.4325190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4325304Z outputs = self.rel_attn( 2025-12-04T09:53:14.4325840Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4325965Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4326543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.4326767Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.4326782Z 2025-12-04T09:53:14.4326964Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4327358Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4327478Z res = mod(**inputs) 2025-12-04T09:53:14.4328027Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4328171Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4328712Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4328824Z outputs = layer_module( 2025-12-04T09:53:14.4329370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4329485Z outputs = self.rel_attn( 2025-12-04T09:53:14.4330021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4330174Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4330755Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4330959Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4330974Z 2025-12-04T09:53:14.4331154Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4331549Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4331664Z res = mod(**inputs) 2025-12-04T09:53:14.4332214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4332350Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4332905Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4333020Z outputs = layer_module( 2025-12-04T09:53:14.4333565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4333744Z outputs = self.rel_attn( 2025-12-04T09:53:14.4334277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4334455Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4335036Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4335265Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4335281Z 2025-12-04T09:53:14.4335414Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4335593Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4335998Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4336102Z res = mod(**inputs) 2025-12-04T09:53:14.4336646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4336796Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4337339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4337457Z outputs = layer_module( 2025-12-04T09:53:14.4337997Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.4338412Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.4338973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.4339098Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.4339649Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.4339773Z output_x = self.ff(output_x) 2025-12-04T09:53:14.4340317Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.4340467Z output = self.activation_function(output) 2025-12-04T09:53:14.4340915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.4341024Z return self.act(input) 2025-12-04T09:53:14.4341046Z 2025-12-04T09:53:14.4341182Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4341362Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4341769Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4341875Z res = mod(**inputs) 2025-12-04T09:53:14.4342420Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4342569Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4343109Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4343221Z outputs = layer_module( 2025-12-04T09:53:14.4343770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4343882Z outputs = self.rel_attn( 2025-12-04T09:53:14.4344430Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 416, in forward 2025-12-04T09:53:14.4344595Z q_head_h = torch.einsum("ibh,hnd->ibnd", h, self.q) 2025-12-04T09:53:14.4344610Z 2025-12-04T09:53:14.4344788Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4345194Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4345383Z res = mod(**inputs) 2025-12-04T09:53:14.4345933Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4346069Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4346638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4346849Z outputs = layer_module( 2025-12-04T09:53:14.4347612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4347734Z outputs = self.rel_attn( 2025-12-04T09:53:14.4348304Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 417, in forward 2025-12-04T09:53:14.4348475Z k_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.k) 2025-12-04T09:53:14.4348490Z 2025-12-04T09:53:14.4348689Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4349103Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4349209Z res = mod(**inputs) 2025-12-04T09:53:14.4349788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4349932Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4350506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4350622Z outputs = layer_module( 2025-12-04T09:53:14.4351178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4351304Z outputs = self.rel_attn( 2025-12-04T09:53:14.4351860Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4351987Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4352587Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 263, in rel_attn_core 2025-12-04T09:53:14.4352834Z ac = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_w_bias, k_head_h) 2025-12-04T09:53:14.4352849Z 2025-12-04T09:53:14.4353043Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4353458Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4353566Z res = mod(**inputs) 2025-12-04T09:53:14.4354140Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4354281Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4354850Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4354968Z outputs = layer_module( 2025-12-04T09:53:14.4355526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4355652Z outputs = self.rel_attn( 2025-12-04T09:53:14.4356211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 422, in forward 2025-12-04T09:53:14.4356464Z k_head_r = torch.einsum("ibh,hnd->ibnd", r.type(self.r.dtype), self.r) 2025-12-04T09:53:14.4356479Z 2025-12-04T09:53:14.4356679Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4357090Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4357203Z res = mod(**inputs) 2025-12-04T09:53:14.4357763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4357976Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4358548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4358662Z outputs = layer_module( 2025-12-04T09:53:14.4359253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4359481Z outputs = self.rel_attn( 2025-12-04T09:53:14.4360047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4360174Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4360753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 266, in rel_attn_core 2025-12-04T09:53:14.4360989Z bd = torch.einsum("ibnd,jbnd->bnij", q_head + self.r_r_bias, k_head_r) 2025-12-04T09:53:14.4361004Z 2025-12-04T09:53:14.4361203Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4361605Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4361721Z res = mod(**inputs) 2025-12-04T09:53:14.4362277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4362418Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4362977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4363092Z outputs = layer_module( 2025-12-04T09:53:14.4363627Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4363748Z outputs = self.rel_attn( 2025-12-04T09:53:14.4364289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 418, in forward 2025-12-04T09:53:14.4364467Z v_head_h = torch.einsum("ibh,hnd->ibnd", cat, self.v) 2025-12-04T09:53:14.4364482Z 2025-12-04T09:53:14.4364663Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4365063Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4365171Z res = mod(**inputs) 2025-12-04T09:53:14.4365719Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4365865Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4366409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4366519Z outputs = layer_module( 2025-12-04T09:53:14.4367059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4367177Z outputs = self.rel_attn( 2025-12-04T09:53:14.4367711Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 425, in forward 2025-12-04T09:53:14.4367836Z attn_vec = self.rel_attn_core( 2025-12-04T09:53:14.4368415Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 294, in rel_attn_core 2025-12-04T09:53:14.4368637Z attn_vec = torch.einsum("bnij,jbnd->ibnd", attn_prob, v_head_h) 2025-12-04T09:53:14.4368652Z 2025-12-04T09:53:14.4368836Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4369232Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4369344Z res = mod(**inputs) 2025-12-04T09:53:14.4369887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4370031Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4370629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4370739Z outputs = layer_module( 2025-12-04T09:53:14.4371309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4371422Z outputs = self.rel_attn( 2025-12-04T09:53:14.4371993Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4372148Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4372729Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4372933Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4372948Z 2025-12-04T09:53:14.4373127Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4373651Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4373781Z res = mod(**inputs) 2025-12-04T09:53:14.4374621Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4374771Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4375333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4375447Z outputs = layer_module( 2025-12-04T09:53:14.4376009Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 494, in forward 2025-12-04T09:53:14.4376128Z outputs = self.rel_attn( 2025-12-04T09:53:14.4376681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 440, in forward 2025-12-04T09:53:14.4376842Z output_h = self.post_attention(h, attn_vec) 2025-12-04T09:53:14.4377440Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 304, in post_attention 2025-12-04T09:53:14.4377647Z attn_out = torch.einsum("ibnd,hnd->ibh", attn_vec, self.o) 2025-12-04T09:53:14.4377662Z 2025-12-04T09:53:14.4377802Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4377987Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4378406Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4378544Z res = mod(**inputs) 2025-12-04T09:53:14.4379133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1616, in forward 2025-12-04T09:53:14.4379285Z transformer_outputs = self.transformer( 2025-12-04T09:53:14.4379842Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1368, in forward 2025-12-04T09:53:14.4379967Z outputs = layer_module( 2025-12-04T09:53:14.4380522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 512, in forward 2025-12-04T09:53:14.4380951Z output_h = apply_chunking_to_forward(self.ff_chunk, self.chunk_size_feed_forward, self.seq_len_dim, output_h) 2025-12-04T09:53:14.4381532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:53:14.4381666Z return forward_fn(*input_tensors) 2025-12-04T09:53:14.4382230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 518, in ff_chunk 2025-12-04T09:53:14.4382351Z output_x = self.ff(output_x) 2025-12-04T09:53:14.4382908Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 464, in forward 2025-12-04T09:53:14.4383134Z output = self.activation_function(output) 2025-12-04T09:53:14.4383590Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:53:14.4383704Z return self.act(input) 2025-12-04T09:53:14.4383729Z 2025-12-04T09:53:14.4383895Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4384033Z cudagraph partition due to non gpu ops 2025-12-04T09:53:14.4384226Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:53:14.4384665Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:53:14.4384771Z res = mod(**inputs) 2025-12-04T09:53:14.4385340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/xlnet/modeling_xlnet.py", line 1639, in forward 2025-12-04T09:53:14.4385570Z loss = loss_fct(logits.view(-1, logits.size(-1)), labels.view(-1)) 2025-12-04T09:53:14.4385586Z 2025-12-04T09:53:25.8512773Z Compilation time (from dynamo_timed): 51.7756174 2025-12-04T09:53:25.8603872Z pass 2025-12-04T09:53:25.8604459Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:53:25.8611067Z TIMING: _recursive_pre_grad_passes:0.09897 _recursive_joint_graph_passes:2.26952 _recursive_post_grad_passes:0.24007 async_compile.wait:0.89869 code_gen:9.82464 inductor_compile:19.13893 backend_compile:42.11636 gc:0.00068 entire_frame_compile:51.77562 total_wall_time:51.77562 2025-12-04T09:53:25.8620853Z STATS: call_* op count: 818 | FakeTensorMode.__torch_dispatch__:68213 | FakeTensor.__torch_dispatch__:13735 | ProxyTorchDispatchMode.__torch_dispatch__:6838 2025-12-04T09:53:25.8621930Z Dynamo produced 1 graphs covering 818 ops with 0 graph breaks (0 unique) 2025-12-04T09:53:29.5935790Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/cuda/__init__.py:65: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you. 2025-12-04T09:53:29.5938800Z import pynvml # type: ignore[import] 2025-12-04T09:53:34.6176456Z 2025-12-04T09:53:36.7949178Z loading model: 0it [00:00, ?it/s] 2025-12-04T09:53:36.7949677Z loading model: 0it [00:02, ?it/s] 2025-12-04T09:53:36.7975302Z cpu eval YituTechConvBert 2025-12-04T09:53:38.1151631Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:53:38.5307118Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:53:38.9458269Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:54:05.2005348Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2005867Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2006331Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2006789Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2008005Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2008552Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2009010Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2009474Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2010360Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2011146Z res = mod(**inputs) 2025-12-04T09:54:05.2011980Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2013002Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2013994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2014989Z hidden_states = self.encoder( 2025-12-04T09:54:05.2015925Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2017291Z layer_outputs = layer_module( 2025-12-04T09:54:05.2018024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2019029Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2020019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2021187Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2022214Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2023208Z self_outputs = self.self( 2025-12-04T09:54:05.2024061Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2025171Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2026393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.2027449Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.2027906Z 2025-12-04T09:54:05.2028106Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2028961Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2029755Z res = mod(**inputs) 2025-12-04T09:54:05.2030547Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2031564Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2032565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2033602Z hidden_states = self.encoder( 2025-12-04T09:54:05.2034447Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2035488Z layer_outputs = layer_module( 2025-12-04T09:54:05.2036334Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2037128Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2038129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2039138Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2040234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2041193Z self_outputs = self.self( 2025-12-04T09:54:05.2046223Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2047477Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2048639Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.2049505Z x = self.pointwise(x) 2025-12-04T09:54:05.2049725Z 2025-12-04T09:54:05.2049868Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2050324Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2051060Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2051806Z res = mod(**inputs) 2025-12-04T09:54:05.2052758Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2053640Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2054788Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2055780Z hidden_states = self.encoder( 2025-12-04T09:54:05.2056832Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2057778Z layer_outputs = layer_module( 2025-12-04T09:54:05.2058565Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2059460Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2060450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2061434Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2062347Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2063349Z self_outputs = self.self( 2025-12-04T09:54:05.2064326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.2065390Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.2065830Z 2025-12-04T09:54:05.2065981Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2066387Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2067072Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2068132Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2068903Z res = mod(**inputs) 2025-12-04T09:54:05.2069791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2070758Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2071763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2072719Z hidden_states = self.encoder( 2025-12-04T09:54:05.2073615Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2074642Z layer_outputs = layer_module( 2025-12-04T09:54:05.2075370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2076250Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2077215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2078256Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2079253Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2080178Z self_outputs = self.self( 2025-12-04T09:54:05.2081133Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.2082249Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.2082614Z 2025-12-04T09:54:05.2082772Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2083289Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2084078Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2084834Z res = mod(**inputs) 2025-12-04T09:54:05.2085610Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2086640Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2087614Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2088608Z hidden_states = self.encoder( 2025-12-04T09:54:05.2089562Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2090567Z layer_outputs = layer_module( 2025-12-04T09:54:05.2091277Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2092226Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2093231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2094117Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2095135Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2096099Z self_outputs = self.self( 2025-12-04T09:54:05.2097056Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.2098103Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.2098445Z 2025-12-04T09:54:05.2098601Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2098979Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2099550Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2100298Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2101778Z res = mod(**inputs) 2025-12-04T09:54:05.2102812Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2103866Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2104937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2105884Z hidden_states = self.encoder( 2025-12-04T09:54:05.2106810Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2107682Z layer_outputs = layer_module( 2025-12-04T09:54:05.2108403Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2109156Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2110029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.2110921Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.2111774Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.2112627Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.2113526Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.2114642Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.2115708Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.2116654Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.2117435Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.2118157Z return self.act(input) 2025-12-04T09:54:05.2118364Z 2025-12-04T09:54:05.2118519Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2119009Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2119383Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2119934Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2120311Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2120679Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2121057Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2121502Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2121923Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2122649Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2123302Z res = mod(**inputs) 2025-12-04T09:54:05.2124104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2124986Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2125831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2126672Z hidden_states = self.encoder( 2025-12-04T09:54:05.2127476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2128316Z layer_outputs = layer_module( 2025-12-04T09:54:05.2129021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2129762Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2130585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2131449Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2132301Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2133124Z self_outputs = self.self( 2025-12-04T09:54:05.2133917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2134936Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2135965Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.2136800Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.2137062Z 2025-12-04T09:54:05.2137252Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2137974Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2138628Z res = mod(**inputs) 2025-12-04T09:54:05.2139381Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2140246Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2141094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2141927Z hidden_states = self.encoder( 2025-12-04T09:54:05.2142742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2143584Z layer_outputs = layer_module( 2025-12-04T09:54:05.2144280Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2144999Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2145841Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2146804Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2147826Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2148686Z self_outputs = self.self( 2025-12-04T09:54:05.2149582Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2150617Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2151690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.2152551Z x = self.pointwise(x) 2025-12-04T09:54:05.2152766Z 2025-12-04T09:54:05.2152940Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2153390Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2154115Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2154784Z res = mod(**inputs) 2025-12-04T09:54:05.2155569Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2156445Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2157314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2158181Z hidden_states = self.encoder( 2025-12-04T09:54:05.2159119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2159945Z layer_outputs = layer_module( 2025-12-04T09:54:05.2160646Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2161389Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2162230Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2163075Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2163928Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2164766Z self_outputs = self.self( 2025-12-04T09:54:05.2165546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.2166469Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.2166819Z 2025-12-04T09:54:05.2166960Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2167710Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2168279Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2169020Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2169689Z res = mod(**inputs) 2025-12-04T09:54:05.2170471Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2171419Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2172287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2173153Z hidden_states = self.encoder( 2025-12-04T09:54:05.2173969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2174885Z layer_outputs = layer_module( 2025-12-04T09:54:05.2175618Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2176369Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2177235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2178112Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2179080Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2180043Z self_outputs = self.self( 2025-12-04T09:54:05.2180872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.2181810Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.2182168Z 2025-12-04T09:54:05.2182318Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2182773Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2183487Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2184133Z res = mod(**inputs) 2025-12-04T09:54:05.2184883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2185745Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2186589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2187783Z hidden_states = self.encoder( 2025-12-04T09:54:05.2188609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2189470Z layer_outputs = layer_module( 2025-12-04T09:54:05.2190190Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2190946Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2191800Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2192684Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2193557Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2194411Z self_outputs = self.self( 2025-12-04T09:54:05.2195222Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.2196154Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.2196487Z 2025-12-04T09:54:05.2196639Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2197024Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2197472Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2198217Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2198974Z res = mod(**inputs) 2025-12-04T09:54:05.2199739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2200600Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2202026Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2202885Z hidden_states = self.encoder( 2025-12-04T09:54:05.2203725Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2204593Z layer_outputs = layer_module( 2025-12-04T09:54:05.2205302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2206072Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2206935Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.2207816Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.2208658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.2209661Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.2210629Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.2211662Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.2212609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.2213595Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.2214492Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.2215196Z return self.act(input) 2025-12-04T09:54:05.2215398Z 2025-12-04T09:54:05.2215536Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2215925Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2216314Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2216682Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2217066Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2217447Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2217813Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2218188Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2218614Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2219313Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2219952Z res = mod(**inputs) 2025-12-04T09:54:05.2220713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2221570Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2222396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2223235Z hidden_states = self.encoder( 2025-12-04T09:54:05.2224034Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2224870Z layer_outputs = layer_module( 2025-12-04T09:54:05.2225551Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2226293Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2227393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2228266Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2229137Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2234831Z self_outputs = self.self( 2025-12-04T09:54:05.2235658Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2236690Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2237747Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.2238616Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.2238869Z 2025-12-04T09:54:05.2239078Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2239803Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2240477Z res = mod(**inputs) 2025-12-04T09:54:05.2241267Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2242238Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2243215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2244061Z hidden_states = self.encoder( 2025-12-04T09:54:05.2244903Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2245727Z layer_outputs = layer_module( 2025-12-04T09:54:05.2246451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2247195Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2248018Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2248881Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2249723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2250560Z self_outputs = self.self( 2025-12-04T09:54:05.2251340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2252348Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2253373Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.2254204Z x = self.pointwise(x) 2025-12-04T09:54:05.2254400Z 2025-12-04T09:54:05.2254537Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2254970Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2255680Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2256314Z res = mod(**inputs) 2025-12-04T09:54:05.2257082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2257942Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2258786Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2259604Z hidden_states = self.encoder( 2025-12-04T09:54:05.2260404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2261235Z layer_outputs = layer_module( 2025-12-04T09:54:05.2261924Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2262649Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2263481Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2264340Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2265169Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2265999Z self_outputs = self.self( 2025-12-04T09:54:05.2266896Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.2268015Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.2268368Z 2025-12-04T09:54:05.2268514Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2268912Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2269356Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2270075Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2270746Z res = mod(**inputs) 2025-12-04T09:54:05.2271531Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2273890Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2274785Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2275645Z hidden_states = self.encoder( 2025-12-04T09:54:05.2276478Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2277366Z layer_outputs = layer_module( 2025-12-04T09:54:05.2278087Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2278850Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2279819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2280675Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2281521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2282356Z self_outputs = self.self( 2025-12-04T09:54:05.2283154Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.2284078Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.2284450Z 2025-12-04T09:54:05.2284592Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2285032Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2285736Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2286384Z res = mod(**inputs) 2025-12-04T09:54:05.2287148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2288020Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2288851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2289700Z hidden_states = self.encoder( 2025-12-04T09:54:05.2290502Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2291338Z layer_outputs = layer_module( 2025-12-04T09:54:05.2297071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2297850Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2298726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2299599Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2300482Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2301745Z self_outputs = self.self( 2025-12-04T09:54:05.2302576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.2303495Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.2303840Z 2025-12-04T09:54:05.2303983Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2304382Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2304811Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2305543Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2306209Z res = mod(**inputs) 2025-12-04T09:54:05.2307111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2308131Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2308999Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2309901Z hidden_states = self.encoder( 2025-12-04T09:54:05.2310734Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2311576Z layer_outputs = layer_module( 2025-12-04T09:54:05.2312339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2313106Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2313953Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.2314835Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.2315689Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.2316531Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.2317437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.2318567Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.2319494Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.2320414Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.2321168Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.2321866Z return self.act(input) 2025-12-04T09:54:05.2322066Z 2025-12-04T09:54:05.2322215Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2322593Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2322975Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2323352Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2323714Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2324095Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2324473Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2324850Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2325266Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2325982Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2326632Z res = mod(**inputs) 2025-12-04T09:54:05.2327382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2328237Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2329084Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2329920Z hidden_states = self.encoder( 2025-12-04T09:54:05.2330713Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2331543Z layer_outputs = layer_module( 2025-12-04T09:54:05.2332234Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2332960Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2333799Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2334653Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2335491Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2336383Z self_outputs = self.self( 2025-12-04T09:54:05.2337174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2338211Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2339231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.2340094Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.2340352Z 2025-12-04T09:54:05.2340536Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2341249Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2341877Z res = mod(**inputs) 2025-12-04T09:54:05.2342637Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2343498Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2344345Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2345177Z hidden_states = self.encoder( 2025-12-04T09:54:05.2345983Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2346912Z layer_outputs = layer_module( 2025-12-04T09:54:05.2347794Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2348541Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2349404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2350286Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2351150Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2352015Z self_outputs = self.self( 2025-12-04T09:54:05.2352831Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2353887Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2355082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.2356006Z x = self.pointwise(x) 2025-12-04T09:54:05.2356210Z 2025-12-04T09:54:05.2356368Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2356820Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2357545Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2358233Z res = mod(**inputs) 2025-12-04T09:54:05.2359023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2359898Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2360781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2361653Z hidden_states = self.encoder( 2025-12-04T09:54:05.2362493Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2363388Z layer_outputs = layer_module( 2025-12-04T09:54:05.2364506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2365298Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2366529Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2367663Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2368576Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2369439Z self_outputs = self.self( 2025-12-04T09:54:05.2370360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.2371319Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.2371672Z 2025-12-04T09:54:05.2371811Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2372197Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2372759Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2373686Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2374351Z res = mod(**inputs) 2025-12-04T09:54:05.2375411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2376275Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2377127Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2377968Z hidden_states = self.encoder( 2025-12-04T09:54:05.2378763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2379600Z layer_outputs = layer_module( 2025-12-04T09:54:05.2380296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2381021Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2381862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2382723Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2383570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2384391Z self_outputs = self.self( 2025-12-04T09:54:05.2385183Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.2386128Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.2386488Z 2025-12-04T09:54:05.2386693Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2387321Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2388060Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2388729Z res = mod(**inputs) 2025-12-04T09:54:05.2389511Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2390398Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2391270Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2392129Z hidden_states = self.encoder( 2025-12-04T09:54:05.2392947Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2393808Z layer_outputs = layer_module( 2025-12-04T09:54:05.2394530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2395282Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2396144Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2397117Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2397988Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2399004Z self_outputs = self.self( 2025-12-04T09:54:05.2399796Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.2400705Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.2401657Z 2025-12-04T09:54:05.2401821Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2402210Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2402657Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2403389Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2404044Z res = mod(**inputs) 2025-12-04T09:54:05.2404843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2405732Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2406606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2407451Z hidden_states = self.encoder( 2025-12-04T09:54:05.2408285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2409145Z layer_outputs = layer_module( 2025-12-04T09:54:05.2409843Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2410605Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2411465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.2412355Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.2413293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.2414113Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.2414998Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.2415993Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.2416904Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.2418176Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.2418975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.2419689Z return self.act(input) 2025-12-04T09:54:05.2419933Z 2025-12-04T09:54:05.2420077Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2420475Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2420854Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2421251Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2421694Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2422075Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2422460Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2422856Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2423284Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2424027Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2424701Z res = mod(**inputs) 2025-12-04T09:54:05.2425549Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2426568Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2427556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2428422Z hidden_states = self.encoder( 2025-12-04T09:54:05.2429293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2430159Z layer_outputs = layer_module( 2025-12-04T09:54:05.2430911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2431674Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2432532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2433419Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2434293Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2435158Z self_outputs = self.self( 2025-12-04T09:54:05.2435964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2437007Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2438065Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.2439013Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.2439269Z 2025-12-04T09:54:05.2439453Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2440164Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2440809Z res = mod(**inputs) 2025-12-04T09:54:05.2441554Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2442411Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2443257Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2444096Z hidden_states = self.encoder( 2025-12-04T09:54:05.2444880Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2445709Z layer_outputs = layer_module( 2025-12-04T09:54:05.2446405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2447130Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2447969Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2448827Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2449672Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2450491Z self_outputs = self.self( 2025-12-04T09:54:05.2451285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2452301Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2453319Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.2454139Z x = self.pointwise(x) 2025-12-04T09:54:05.2454347Z 2025-12-04T09:54:05.2454483Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2454918Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2455619Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2456334Z res = mod(**inputs) 2025-12-04T09:54:05.2457092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2457990Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2458829Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2459707Z hidden_states = self.encoder( 2025-12-04T09:54:05.2460510Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2461334Z layer_outputs = layer_module( 2025-12-04T09:54:05.2462028Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2462763Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2463606Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2464446Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2465287Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2466125Z self_outputs = self.self( 2025-12-04T09:54:05.2467011Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.2468120Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.2468482Z 2025-12-04T09:54:05.2468624Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2469022Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2469454Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2470194Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2470866Z res = mod(**inputs) 2025-12-04T09:54:05.2471651Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2472524Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2473396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2474253Z hidden_states = self.encoder( 2025-12-04T09:54:05.2475067Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2475927Z layer_outputs = layer_module( 2025-12-04T09:54:05.2476642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2477400Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2478252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2479138Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2484178Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2485038Z self_outputs = self.self( 2025-12-04T09:54:05.2485865Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.2486837Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.2487209Z 2025-12-04T09:54:05.2487366Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2487799Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2488532Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2489297Z res = mod(**inputs) 2025-12-04T09:54:05.2490083Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2490959Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2491973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2492814Z hidden_states = self.encoder( 2025-12-04T09:54:05.2493636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2494468Z layer_outputs = layer_module( 2025-12-04T09:54:05.2495165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2495904Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2496732Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2497596Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2498450Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2499290Z self_outputs = self.self( 2025-12-04T09:54:05.2500064Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.2501521Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.2501867Z 2025-12-04T09:54:05.2502037Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2502431Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2502879Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2503625Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2504300Z res = mod(**inputs) 2025-12-04T09:54:05.2505078Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2505966Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2506938Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2507794Z hidden_states = self.encoder( 2025-12-04T09:54:05.2508636Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2509494Z layer_outputs = layer_module( 2025-12-04T09:54:05.2510215Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2510970Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2511838Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.2512730Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.2513572Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.2514420Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.2515339Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.2516368Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.2517305Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.2518241Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.2519117Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.2519963Z return self.act(input) 2025-12-04T09:54:05.2520164Z 2025-12-04T09:54:05.2520300Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2520681Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2521097Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2521465Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2521839Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2522215Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2522621Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2522999Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2523430Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2524145Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2524782Z res = mod(**inputs) 2025-12-04T09:54:05.2525543Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2526406Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2527240Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2528078Z hidden_states = self.encoder( 2025-12-04T09:54:05.2528883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2529723Z layer_outputs = layer_module( 2025-12-04T09:54:05.2530404Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2531143Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2531978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2532844Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2533678Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2534514Z self_outputs = self.self( 2025-12-04T09:54:05.2535307Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2553415Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2554541Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.2555435Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.2555695Z 2025-12-04T09:54:05.2555903Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2556629Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2557309Z res = mod(**inputs) 2025-12-04T09:54:05.2558102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2559091Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2559932Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2560773Z hidden_states = self.encoder( 2025-12-04T09:54:05.2561589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2562410Z layer_outputs = layer_module( 2025-12-04T09:54:05.2563107Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2563853Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2564694Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2565652Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2566561Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2567396Z self_outputs = self.self( 2025-12-04T09:54:05.2568177Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2569209Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2570225Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.2571044Z x = self.pointwise(x) 2025-12-04T09:54:05.2571238Z 2025-12-04T09:54:05.2571373Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2571794Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2572500Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2573135Z res = mod(**inputs) 2025-12-04T09:54:05.2573879Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2574725Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2575570Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2576386Z hidden_states = self.encoder( 2025-12-04T09:54:05.2577166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2577985Z layer_outputs = layer_module( 2025-12-04T09:54:05.2578657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2579381Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2580210Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2581064Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2581892Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2582703Z self_outputs = self.self( 2025-12-04T09:54:05.2583484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.2584395Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.2584729Z 2025-12-04T09:54:05.2584863Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2585239Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2585661Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2586362Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2587294Z res = mod(**inputs) 2025-12-04T09:54:05.2588081Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2588958Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2589807Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2590657Z hidden_states = self.encoder( 2025-12-04T09:54:05.2591477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2592327Z layer_outputs = layer_module( 2025-12-04T09:54:05.2593024Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2593857Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2594714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2595621Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2596477Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2597331Z self_outputs = self.self( 2025-12-04T09:54:05.2598170Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.2599221Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.2599585Z 2025-12-04T09:54:05.2599719Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2600146Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2601449Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2602128Z res = mod(**inputs) 2025-12-04T09:54:05.2602917Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2603795Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2604739Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2605658Z hidden_states = self.encoder( 2025-12-04T09:54:05.2606483Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2607341Z layer_outputs = layer_module( 2025-12-04T09:54:05.2608043Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2608801Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2609657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2610519Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2611372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2612223Z self_outputs = self.self( 2025-12-04T09:54:05.2613021Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.2613933Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.2614270Z 2025-12-04T09:54:05.2614409Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2614786Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2615217Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2615938Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2616597Z res = mod(**inputs) 2025-12-04T09:54:05.2617475Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2618312Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2619148Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2619976Z hidden_states = self.encoder( 2025-12-04T09:54:05.2620777Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2621598Z layer_outputs = layer_module( 2025-12-04T09:54:05.2622278Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2623144Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2623985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.2624833Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.2625696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.2626499Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.2627760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.2628787Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.2629738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.2630666Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.2631444Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.2632163Z return self.act(input) 2025-12-04T09:54:05.2632366Z 2025-12-04T09:54:05.2632519Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2632906Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2633281Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2633669Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2634051Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2634425Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2634797Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2635176Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2635593Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2636321Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2636980Z res = mod(**inputs) 2025-12-04T09:54:05.2637769Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2638636Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2639601Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2640429Z hidden_states = self.encoder( 2025-12-04T09:54:05.2641221Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2642055Z layer_outputs = layer_module( 2025-12-04T09:54:05.2642741Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2643466Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2644284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2645130Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2645964Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2646784Z self_outputs = self.self( 2025-12-04T09:54:05.2647558Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2648564Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2649567Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.2650390Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.2650644Z 2025-12-04T09:54:05.2650829Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2651529Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2652232Z res = mod(**inputs) 2025-12-04T09:54:05.2652977Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2653871Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2654714Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2655568Z hidden_states = self.encoder( 2025-12-04T09:54:05.2656370Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2657193Z layer_outputs = layer_module( 2025-12-04T09:54:05.2657878Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2658599Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2659437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2660279Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2661120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2661932Z self_outputs = self.self( 2025-12-04T09:54:05.2662716Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2663723Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2664726Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.2665546Z x = self.pointwise(x) 2025-12-04T09:54:05.2665757Z 2025-12-04T09:54:05.2665894Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2666318Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2667405Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2668184Z res = mod(**inputs) 2025-12-04T09:54:05.2668975Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2669834Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2670696Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2671537Z hidden_states = self.encoder( 2025-12-04T09:54:05.2672411Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2673265Z layer_outputs = layer_module( 2025-12-04T09:54:05.2673985Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2674739Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2675581Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2676453Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2677314Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2678164Z self_outputs = self.self( 2025-12-04T09:54:05.2678955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.2680011Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.2680348Z 2025-12-04T09:54:05.2680491Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2680942Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2681374Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2682081Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2682748Z res = mod(**inputs) 2025-12-04T09:54:05.2683497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2684344Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2685211Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2686041Z hidden_states = self.encoder( 2025-12-04T09:54:05.2686828Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2687651Z layer_outputs = layer_module( 2025-12-04T09:54:05.2688340Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2689062Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2689889Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2690735Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2691575Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2692386Z self_outputs = self.self( 2025-12-04T09:54:05.2693165Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.2694100Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.2694455Z 2025-12-04T09:54:05.2694599Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2695016Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2695719Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2696352Z res = mod(**inputs) 2025-12-04T09:54:05.2697092Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2697936Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2698766Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2699584Z hidden_states = self.encoder( 2025-12-04T09:54:05.2700372Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2701796Z layer_outputs = layer_module( 2025-12-04T09:54:05.2702508Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2703258Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2704111Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2704982Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2705848Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2706776Z self_outputs = self.self( 2025-12-04T09:54:05.2707585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.2708503Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.2708828Z 2025-12-04T09:54:05.2708975Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2709348Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2709911Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2710635Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2711282Z res = mod(**inputs) 2025-12-04T09:54:05.2712094Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2712963Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2713862Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2714701Z hidden_states = self.encoder( 2025-12-04T09:54:05.2715519Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2716367Z layer_outputs = layer_module( 2025-12-04T09:54:05.2717054Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2717803Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2718753Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.2719601Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.2720406Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.2721219Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.2722097Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.2723081Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.2723987Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.2724884Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.2725642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.2726325Z return self.act(input) 2025-12-04T09:54:05.2726532Z 2025-12-04T09:54:05.2726663Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2727036Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2727403Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2727757Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2728119Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2728670Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2729027Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2729398Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2734965Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2735728Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2736402Z res = mod(**inputs) 2025-12-04T09:54:05.2737186Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2738065Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2738920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2739779Z hidden_states = self.encoder( 2025-12-04T09:54:05.2740594Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2741446Z layer_outputs = layer_module( 2025-12-04T09:54:05.2742249Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2742979Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2743900Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2744736Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2745608Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2746436Z self_outputs = self.self( 2025-12-04T09:54:05.2747521Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2748544Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2749585Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.2750439Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.2750687Z 2025-12-04T09:54:05.2750882Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2751599Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2752253Z res = mod(**inputs) 2025-12-04T09:54:05.2753023Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2753880Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2754738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2755587Z hidden_states = self.encoder( 2025-12-04T09:54:05.2756409Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2757248Z layer_outputs = layer_module( 2025-12-04T09:54:05.2757946Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2758803Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2759622Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2760471Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2761302Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2762115Z self_outputs = self.self( 2025-12-04T09:54:05.2762885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2763871Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2764872Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.2765690Z x = self.pointwise(x) 2025-12-04T09:54:05.2765882Z 2025-12-04T09:54:05.2766014Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2766435Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2767147Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2767775Z res = mod(**inputs) 2025-12-04T09:54:05.2768532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2769378Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2770217Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2771027Z hidden_states = self.encoder( 2025-12-04T09:54:05.2771819Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2772729Z layer_outputs = layer_module( 2025-12-04T09:54:05.2773393Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2774120Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2774978Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2775822Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2776669Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2777486Z self_outputs = self.self( 2025-12-04T09:54:05.2778262Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.2779174Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.2779509Z 2025-12-04T09:54:05.2779640Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2780010Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2780424Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2781119Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2781756Z res = mod(**inputs) 2025-12-04T09:54:05.2782499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2783341Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2784159Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2784982Z hidden_states = self.encoder( 2025-12-04T09:54:05.2785775Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2786589Z layer_outputs = layer_module( 2025-12-04T09:54:05.2787532Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2788280Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2789129Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2789991Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2790845Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2791693Z self_outputs = self.self( 2025-12-04T09:54:05.2797499Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.2798451Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.2798832Z 2025-12-04T09:54:05.2798970Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2799407Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2800119Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2801173Z res = mod(**inputs) 2025-12-04T09:54:05.2801974Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2802851Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2803705Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2804561Z hidden_states = self.encoder( 2025-12-04T09:54:05.2805382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2806230Z layer_outputs = layer_module( 2025-12-04T09:54:05.2807059Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2807810Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2808695Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2809550Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2810449Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2811299Z self_outputs = self.self( 2025-12-04T09:54:05.2812113Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.2813126Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.2813451Z 2025-12-04T09:54:05.2813584Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2813965Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2814376Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2815074Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2815710Z res = mod(**inputs) 2025-12-04T09:54:05.2816465Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2817301Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2818136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2818952Z hidden_states = self.encoder( 2025-12-04T09:54:05.2819736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2820561Z layer_outputs = layer_module( 2025-12-04T09:54:05.2821246Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2821969Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2822787Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.2823628Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.2824443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.2825256Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.2826119Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.2827361Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.2828309Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.2829227Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.2830019Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.2830736Z return self.act(input) 2025-12-04T09:54:05.2830938Z 2025-12-04T09:54:05.2831084Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2831460Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2831838Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2832216Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2832580Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2832949Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2833320Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2833687Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2834124Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2834923Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2835582Z res = mod(**inputs) 2025-12-04T09:54:05.2836395Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2837267Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2838158Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2839127Z hidden_states = self.encoder( 2025-12-04T09:54:05.2839920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2840741Z layer_outputs = layer_module( 2025-12-04T09:54:05.2841437Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2842170Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2843005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2843862Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2844699Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2845518Z self_outputs = self.self( 2025-12-04T09:54:05.2846308Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2847311Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2848329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.2849155Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.2849411Z 2025-12-04T09:54:05.2849595Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2850298Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2850934Z res = mod(**inputs) 2025-12-04T09:54:05.2851693Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2852545Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2853379Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2854196Z hidden_states = self.encoder( 2025-12-04T09:54:05.2855365Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2856216Z layer_outputs = layer_module( 2025-12-04T09:54:05.2856915Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2857658Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2858512Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2859385Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2860245Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2861089Z self_outputs = self.self( 2025-12-04T09:54:05.2861887Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2862919Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2863956Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.2864892Z x = self.pointwise(x) 2025-12-04T09:54:05.2865093Z 2025-12-04T09:54:05.2865239Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2865699Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2866422Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2867160Z res = mod(**inputs) 2025-12-04T09:54:05.2867973Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2868843Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2869702Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2870559Z hidden_states = self.encoder( 2025-12-04T09:54:05.2871382Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2872223Z layer_outputs = layer_module( 2025-12-04T09:54:05.2872930Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2873693Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2874546Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2875419Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2876282Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2877130Z self_outputs = self.self( 2025-12-04T09:54:05.2877927Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.2878962Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.2879294Z 2025-12-04T09:54:05.2879438Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2879803Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2880222Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2880926Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2881566Z res = mod(**inputs) 2025-12-04T09:54:05.2882311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2883156Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2883990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2884819Z hidden_states = self.encoder( 2025-12-04T09:54:05.2885603Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2886432Z layer_outputs = layer_module( 2025-12-04T09:54:05.2887118Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2887842Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2888676Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2889522Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2890355Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2891166Z self_outputs = self.self( 2025-12-04T09:54:05.2891952Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.2892950Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.2893306Z 2025-12-04T09:54:05.2893452Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2893868Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2894595Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2895231Z res = mod(**inputs) 2025-12-04T09:54:05.2896005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2896848Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2897681Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2898507Z hidden_states = self.encoder( 2025-12-04T09:54:05.2899289Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2900115Z layer_outputs = layer_module( 2025-12-04T09:54:05.2901333Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2902096Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2902955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2903827Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2904692Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2905534Z self_outputs = self.self( 2025-12-04T09:54:05.2906341Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.2907347Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.2907683Z 2025-12-04T09:54:05.2907835Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2908215Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2908653Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2909386Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2910033Z res = mod(**inputs) 2025-12-04T09:54:05.2910821Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2911705Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2912560Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2913406Z hidden_states = self.encoder( 2025-12-04T09:54:05.2914231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2915094Z layer_outputs = layer_module( 2025-12-04T09:54:05.2915791Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2916541Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2917500Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.2918432Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.2919268Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.2920107Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.2921016Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.2922029Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.2923174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.2924107Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.2924940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.2925647Z return self.act(input) 2025-12-04T09:54:05.2925861Z 2025-12-04T09:54:05.2926003Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2926435Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2926818Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2927184Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2927557Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2927939Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2928306Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2928678Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2929229Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2929924Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2930558Z res = mod(**inputs) 2025-12-04T09:54:05.2931311Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2932160Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2932989Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2933811Z hidden_states = self.encoder( 2025-12-04T09:54:05.2934609Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2935421Z layer_outputs = layer_module( 2025-12-04T09:54:05.2936101Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2936831Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2937656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2938495Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2939329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2940157Z self_outputs = self.self( 2025-12-04T09:54:05.2940937Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2941943Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2942955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.2943789Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.2944032Z 2025-12-04T09:54:05.2944215Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2944914Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2945556Z res = mod(**inputs) 2025-12-04T09:54:05.2946310Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2947443Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2948313Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2949166Z hidden_states = self.encoder( 2025-12-04T09:54:05.2949972Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2950892Z layer_outputs = layer_module( 2025-12-04T09:54:05.2951598Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2952353Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2953238Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2954109Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2955010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2955865Z self_outputs = self.self( 2025-12-04T09:54:05.2956662Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.2957693Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.2958742Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.2959680Z x = self.pointwise(x) 2025-12-04T09:54:05.2959870Z 2025-12-04T09:54:05.2960011Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2960435Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2961140Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2961768Z res = mod(**inputs) 2025-12-04T09:54:05.2962522Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2963367Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2964209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2965025Z hidden_states = self.encoder( 2025-12-04T09:54:05.2965823Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2966647Z layer_outputs = layer_module( 2025-12-04T09:54:05.2967326Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2968054Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2968881Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2969729Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2970556Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2971390Z self_outputs = self.self( 2025-12-04T09:54:05.2972166Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.2973075Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.2973406Z 2025-12-04T09:54:05.2973539Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2973909Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2974331Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2975019Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2975649Z res = mod(**inputs) 2025-12-04T09:54:05.2976405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2977251Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2978082Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2978904Z hidden_states = self.encoder( 2025-12-04T09:54:05.2984814Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.2985730Z layer_outputs = layer_module( 2025-12-04T09:54:05.2986488Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.2987344Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.2988269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.2989139Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.2990005Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.2990854Z self_outputs = self.self( 2025-12-04T09:54:05.2991660Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.2992626Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.2993000Z 2025-12-04T09:54:05.2993139Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.2993584Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.2994300Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.2994957Z res = mod(**inputs) 2025-12-04T09:54:05.2995736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.2996597Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.2997443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.2998406Z hidden_states = self.encoder( 2025-12-04T09:54:05.2999197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3000006Z layer_outputs = layer_module( 2025-12-04T09:54:05.3000690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3001973Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3002837Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3003703Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3004564Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3005401Z self_outputs = self.self( 2025-12-04T09:54:05.3006197Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.3007119Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.3007453Z 2025-12-04T09:54:05.3007588Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3007967Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3008391Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3009113Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3009765Z res = mod(**inputs) 2025-12-04T09:54:05.3010539Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3011416Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3012284Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3013141Z hidden_states = self.encoder( 2025-12-04T09:54:05.3014188Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3015006Z layer_outputs = layer_module( 2025-12-04T09:54:05.3015728Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3016449Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3017265Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.3018151Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.3018968Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.3019768Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.3020642Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.3021634Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.3022548Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.3023452Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.3024205Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.3024886Z return self.act(input) 2025-12-04T09:54:05.3025086Z 2025-12-04T09:54:05.3025228Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3025592Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3025958Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3026324Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3026768Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3027310Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3027707Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3028075Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3028512Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3029247Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3029898Z res = mod(**inputs) 2025-12-04T09:54:05.3030667Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3031548Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3032407Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3033242Z hidden_states = self.encoder( 2025-12-04T09:54:05.3034060Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3034914Z layer_outputs = layer_module( 2025-12-04T09:54:05.3035611Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3036357Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3037209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3038078Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3038942Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3039867Z self_outputs = self.self( 2025-12-04T09:54:05.3040641Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.3041643Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.3047723Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.3048586Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.3048844Z 2025-12-04T09:54:05.3049073Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3049795Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3050447Z res = mod(**inputs) 2025-12-04T09:54:05.3051252Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3052135Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3052994Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3053841Z hidden_states = self.encoder( 2025-12-04T09:54:05.3054657Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3055513Z layer_outputs = layer_module( 2025-12-04T09:54:05.3056212Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3056957Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3057809Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3058826Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3059656Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3060476Z self_outputs = self.self( 2025-12-04T09:54:05.3061254Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.3062251Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.3063239Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.3064059Z x = self.pointwise(x) 2025-12-04T09:54:05.3064252Z 2025-12-04T09:54:05.3064392Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3064808Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3065511Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3066151Z res = mod(**inputs) 2025-12-04T09:54:05.3066990Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3068022Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3068883Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3069740Z hidden_states = self.encoder( 2025-12-04T09:54:05.3070552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3071403Z layer_outputs = layer_module( 2025-12-04T09:54:05.3072102Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3072854Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3073697Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3074567Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3075425Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3076275Z self_outputs = self.self( 2025-12-04T09:54:05.3077136Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.3078079Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.3078425Z 2025-12-04T09:54:05.3078602Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3079090Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3079511Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3080235Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3080870Z res = mod(**inputs) 2025-12-04T09:54:05.3081612Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3082453Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3083275Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3084095Z hidden_states = self.encoder( 2025-12-04T09:54:05.3084885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3085702Z layer_outputs = layer_module( 2025-12-04T09:54:05.3086383Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3087097Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3087922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3088758Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3089574Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3090391Z self_outputs = self.self( 2025-12-04T09:54:05.3091179Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.3092108Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.3092464Z 2025-12-04T09:54:05.3092599Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3093018Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3093714Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3094348Z res = mod(**inputs) 2025-12-04T09:54:05.3095089Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3095928Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3096762Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3097585Z hidden_states = self.encoder( 2025-12-04T09:54:05.3098378Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3099199Z layer_outputs = layer_module( 2025-12-04T09:54:05.3099885Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3100605Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3102008Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3102898Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3103776Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3104716Z self_outputs = self.self( 2025-12-04T09:54:05.3105600Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.3106758Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.3107105Z 2025-12-04T09:54:05.3107249Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3107708Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3108157Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3108894Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3119551Z res = mod(**inputs) 2025-12-04T09:54:05.3120443Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3121294Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3122134Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3122970Z hidden_states = self.encoder( 2025-12-04T09:54:05.3123760Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3124582Z layer_outputs = layer_module( 2025-12-04T09:54:05.3125269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3125996Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3126818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.3127666Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.3128476Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.3129283Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.3130152Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.3131130Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.3132046Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.3132938Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.3133683Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.3134369Z return self.act(input) 2025-12-04T09:54:05.3134568Z 2025-12-04T09:54:05.3134707Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3135073Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3135436Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3135796Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3136151Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3136515Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3136877Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3137245Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3137659Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3138362Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3138990Z res = mod(**inputs) 2025-12-04T09:54:05.3139736Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3140578Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3141405Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3142229Z hidden_states = self.encoder( 2025-12-04T09:54:05.3143010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3143912Z layer_outputs = layer_module( 2025-12-04T09:54:05.3144599Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3145347Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3146171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3147338Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3148231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3149069Z self_outputs = self.self( 2025-12-04T09:54:05.3149870Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.3150898Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.3151940Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 281, in forward 2025-12-04T09:54:05.3152792Z x = self.depthwise(hidden_states) 2025-12-04T09:54:05.3153055Z 2025-12-04T09:54:05.3153243Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3153962Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3154606Z res = mod(**inputs) 2025-12-04T09:54:05.3155384Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3156249Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3157104Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3157945Z hidden_states = self.encoder( 2025-12-04T09:54:05.3158761Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3159699Z layer_outputs = layer_module( 2025-12-04T09:54:05.3160375Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3161100Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3161922Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3162760Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3163579Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3164394Z self_outputs = self.self( 2025-12-04T09:54:05.3165171Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 346, in forward 2025-12-04T09:54:05.3166166Z mixed_key_conv_attn_layer = self.key_conv_attn_layer(hidden_states.transpose(1, 2)) 2025-12-04T09:54:05.3167324Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 282, in forward 2025-12-04T09:54:05.3168430Z x = self.pointwise(x) 2025-12-04T09:54:05.3168627Z 2025-12-04T09:54:05.3168770Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3169194Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3169925Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3170575Z res = mod(**inputs) 2025-12-04T09:54:05.3171349Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3172212Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3173216Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3174065Z hidden_states = self.encoder( 2025-12-04T09:54:05.3174918Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3175753Z layer_outputs = layer_module( 2025-12-04T09:54:05.3176451Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3177227Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3178070Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3178946Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3179907Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3180733Z self_outputs = self.self( 2025-12-04T09:54:05.3181497Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 361, in forward 2025-12-04T09:54:05.3182413Z conv_kernel_layer = self.conv_kernel_layer(conv_attn_layer) 2025-12-04T09:54:05.3182744Z 2025-12-04T09:54:05.3182890Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3183254Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3183672Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3184383Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3185018Z res = mod(**inputs) 2025-12-04T09:54:05.3185763Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3186614Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3187744Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3188597Z hidden_states = self.encoder( 2025-12-04T09:54:05.3189408Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3190257Z layer_outputs = layer_module( 2025-12-04T09:54:05.3190738Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3190886Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3191484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3191631Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3192235Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3192356Z self_outputs = self.self( 2025-12-04T09:54:05.3192955Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 379, in forward 2025-12-04T09:54:05.3193185Z conv_out_layer = torch.matmul(conv_out_layer, conv_kernel_layer) 2025-12-04T09:54:05.3193201Z 2025-12-04T09:54:05.3193335Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3193528Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3193941Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3194054Z res = mod(**inputs) 2025-12-04T09:54:05.3194653Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3194797Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3195396Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3195585Z hidden_states = self.encoder( 2025-12-04T09:54:05.3196174Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3196330Z layer_outputs = layer_module( 2025-12-04T09:54:05.3196818Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3196990Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3197589Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 560, in forward 2025-12-04T09:54:05.3197731Z self_attention_outputs = self.attention( 2025-12-04T09:54:05.3198328Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 463, in forward 2025-12-04T09:54:05.3198450Z self_outputs = self.self( 2025-12-04T09:54:05.3199047Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 404, in forward 2025-12-04T09:54:05.3199350Z context_layer = torch.cat([context_layer, conv_out], 2) 2025-12-04T09:54:05.3199366Z 2025-12-04T09:54:05.3199498Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3199634Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3199814Z cudagraph partition due to non gpu ops. Found from : 2025-12-04T09:54:05.3200212Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/huggingface.py", line 553, in forward_pass 2025-12-04T09:54:05.3200320Z res = mod(**inputs) 2025-12-04T09:54:05.3201285Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 924, in forward 2025-12-04T09:54:05.3201608Z generator_hidden_states = self.convbert( 2025-12-04T09:54:05.3202209Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 852, in forward 2025-12-04T09:54:05.3202350Z hidden_states = self.encoder( 2025-12-04T09:54:05.3202960Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 624, in forward 2025-12-04T09:54:05.3203085Z layer_outputs = layer_module( 2025-12-04T09:54:05.3203566Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/modeling_layers.py", line 94, in __call__ 2025-12-04T09:54:05.3203708Z return super().__call__(*args, **kwargs) 2025-12-04T09:54:05.3204296Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 585, in forward 2025-12-04T09:54:05.3204445Z layer_output = apply_chunking_to_forward( 2025-12-04T09:54:05.3205010Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/pytorch_utils.py", line 257, in apply_chunking_to_forward 2025-12-04T09:54:05.3205141Z return forward_fn(*input_tensors) 2025-12-04T09:54:05.3205806Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 592, in feed_forward_chunk 2025-12-04T09:54:05.3206020Z intermediate_output = self.intermediate(attention_output) 2025-12-04T09:54:05.3206619Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/models/convbert/modeling_convbert.py", line 513, in forward 2025-12-04T09:54:05.3206812Z hidden_states = self.intermediate_act_fn(hidden_states) 2025-12-04T09:54:05.3207269Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/transformers/activations.py", line 85, in forward 2025-12-04T09:54:05.3207391Z return self.act(input) 2025-12-04T09:54:05.3207407Z 2025-12-04T09:54:05.3207540Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3207670Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3207806Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3207937Z cudagraph partition due to non gpu ops 2025-12-04T09:54:05.3208208Z cudagraph partition due to non gpu ops 2025-12-04T09:54:14.0686036Z Compilation time (from dynamo_timed): 34.017516032 2025-12-04T09:54:14.0776104Z pass 2025-12-04T09:54:14.0776888Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2025-12-04T09:54:14.0783691Z TIMING: _recursive_pre_grad_passes:0.33874 _recursive_joint_graph_passes:1.00391 _recursive_post_grad_passes:0.21494 async_compile.wait:1.12 code_gen:9.03206 inductor_compile:14.29977 backend_compile:27.91393 gc:0.00017 entire_frame_compile:34.01752 total_wall_time:34.01752 2025-12-04T09:54:14.0785756Z STATS: call_* op count: 634 | FakeTensorMode.__torch_dispatch__:38075 | FakeTensor.__torch_dispatch__:5654 | ProxyTorchDispatchMode.__torch_dispatch__:5038 2025-12-04T09:54:14.0786953Z Dynamo produced 1 graphs covering 634 ops with 0 graph breaks (0 unique) 2025-12-04T09:54:16.0908893Z accuracy pass_rate=92.59% 2025-12-04T09:54:16.0909421Z calls_captured gmean=0.00x mean=579.519x 2025-12-04T09:54:16.0913212Z unique_graphs gmean=0.00x mean=1.111x 2025-12-04T09:54:16.0917189Z graph_breaks gmean=0.00x mean=0.222x 2025-12-04T09:54:16.0921257Z unique_graph_breaks gmean=0.00x mean=0.074x 2025-12-04T09:54:16.0925138Z autograd_captures gmean=0.00x mean=0.000x 2025-12-04T09:54:16.0928999Z autograd_compiles gmean=0.00x mean=0.000x 2025-12-04T09:54:16.0932820Z cudagraph_skips gmean=0.00x mean=1.111x 2025-12-04T09:54:16.0933982Z compilation_latency mean=28.175 seconds 2025-12-04T09:54:17.0428458Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_huggingface.csv --expected benchmarks/dynamo/ci_expected_accuracy/cpu_inductor_freezing_huggingface_inference.csv 2025-12-04T09:54:17.4689881Z AlbertForMaskedLM PASS 2025-12-04T09:54:17.4690662Z AllenaiLongformerBase PASS 2025-12-04T09:54:17.4696387Z BartForCausalLM PASS 2025-12-04T09:54:17.4703189Z BertForMaskedLM PASS 2025-12-04T09:54:17.4709549Z BlenderbotForCausalLM XFAIL 2025-12-04T09:54:17.4720932Z DebertaV2ForMaskedLM XFAIL 2025-12-04T09:54:17.4726658Z DistilBertForMaskedLM PASS 2025-12-04T09:54:17.4732360Z DistillGPT2 PASS 2025-12-04T09:54:17.4738280Z ElectraForCausalLM PASS 2025-12-04T09:54:17.4743977Z GPT2ForSequenceClassification PASS 2025-12-04T09:54:17.4750180Z GoogleFnet PASS 2025-12-04T09:54:17.4756100Z LayoutLMForMaskedLM PASS 2025-12-04T09:54:17.4762092Z M2M100ForConditionalGeneration PASS 2025-12-04T09:54:17.4767769Z MBartForCausalLM PASS 2025-12-04T09:54:17.4780306Z MT5ForConditionalGeneration PASS 2025-12-04T09:54:17.4780747Z MegatronBertForCausalLM PASS 2025-12-04T09:54:17.4786182Z MobileBertForMaskedLM PASS 2025-12-04T09:54:17.4792298Z OPTForCausalLM PASS 2025-12-04T09:54:17.4798149Z PLBartForCausalLM PASS 2025-12-04T09:54:17.4805119Z PegasusForCausalLM PASS 2025-12-04T09:54:17.4810961Z RobertaForCausalLM PASS 2025-12-04T09:54:17.4816877Z T5ForConditionalGeneration PASS 2025-12-04T09:54:17.4822480Z T5Small PASS 2025-12-04T09:54:17.4828535Z TrOCRForCausalLM PASS 2025-12-04T09:54:17.4834456Z XGLMForCausalLM PASS 2025-12-04T09:54:17.4840949Z XLNetLMHeadModel PASS 2025-12-04T09:54:17.4846580Z YituTechConvBert PASS 2025-12-04T09:54:17.5388396Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_huggingface.csv --expected benchmarks/dynamo/ci_expected_accuracy/cpu_inductor_freezing_huggingface_inference.csv 2025-12-04T09:54:17.9605522Z AlbertForMaskedLM PASS 2025-12-04T09:54:17.9610110Z AllenaiLongformerBase PASS 2025-12-04T09:54:17.9615964Z BartForCausalLM PASS 2025-12-04T09:54:17.9621795Z BertForMaskedLM PASS 2025-12-04T09:54:17.9628076Z BlenderbotForCausalLM PASS 2025-12-04T09:54:17.9634054Z DebertaV2ForMaskedLM PASS 2025-12-04T09:54:17.9640227Z DistilBertForMaskedLM PASS 2025-12-04T09:54:17.9645902Z DistillGPT2 PASS 2025-12-04T09:54:17.9652590Z ElectraForCausalLM PASS 2025-12-04T09:54:17.9658552Z GPT2ForSequenceClassification PASS 2025-12-04T09:54:17.9664703Z GoogleFnet PASS 2025-12-04T09:54:17.9670878Z LayoutLMForMaskedLM PASS 2025-12-04T09:54:17.9676840Z M2M100ForConditionalGeneration PASS 2025-12-04T09:54:17.9682736Z MBartForCausalLM PASS 2025-12-04T09:54:17.9688542Z MT5ForConditionalGeneration PASS 2025-12-04T09:54:17.9694246Z MegatronBertForCausalLM PASS 2025-12-04T09:54:17.9700014Z MobileBertForMaskedLM PASS 2025-12-04T09:54:17.9706977Z OPTForCausalLM PASS 2025-12-04T09:54:17.9718429Z PLBartForCausalLM PASS 2025-12-04T09:54:17.9724471Z PegasusForCausalLM PASS 2025-12-04T09:54:17.9730181Z RobertaForCausalLM PASS 2025-12-04T09:54:17.9735920Z T5ForConditionalGeneration PASS 2025-12-04T09:54:17.9741639Z T5Small PASS 2025-12-04T09:54:17.9747803Z TrOCRForCausalLM PASS 2025-12-04T09:54:17.9753857Z XGLMForCausalLM PASS_BUT_FLAKY 2025-12-04T09:54:17.9759793Z XLNetLMHeadModel PASS 2025-12-04T09:54:17.9765506Z YituTechConvBert PASS 2025-12-04T09:54:18.0283594Z + sccache_epilogue 2025-12-04T09:54:18.0283996Z + echo '::group::Sccache Compilation Log' 2025-12-04T09:54:18.0284760Z ##[group]Sccache Compilation Log 2025-12-04T09:54:18.0285185Z + echo '=================== sccache compilation log ===================' 2025-12-04T09:54:18.0285704Z =================== sccache compilation log =================== 2025-12-04T09:54:18.0286451Z + python /var/lib/jenkins/workspace/.ci/pytorch/print_sccache_log.py /var/lib/jenkins/sccache_error.log 2025-12-04T09:54:18.0613066Z + echo '=========== If your build fails, please take a look at the log above for possible reasons ===========' 2025-12-04T09:54:18.0614022Z =========== If your build fails, please take a look at the log above for possible reasons =========== 2025-12-04T09:54:18.0614599Z + sccache --show-stats 2025-12-04T09:54:18.0652680Z Compile requests 276 2025-12-04T09:54:18.0653108Z Compile requests executed 0 2025-12-04T09:54:18.0653474Z Cache hits 0 2025-12-04T09:54:18.0653838Z Cache misses 0 2025-12-04T09:54:18.0654205Z Cache hits rate - 2025-12-04T09:54:18.0654558Z Cache timeouts 0 2025-12-04T09:54:18.0654921Z Cache read errors 0 2025-12-04T09:54:18.0655296Z Forced recaches 0 2025-12-04T09:54:18.0655661Z Cache write errors 0 2025-12-04T09:54:18.0656006Z Cache errors 0 2025-12-04T09:54:18.0656369Z Compilations 0 2025-12-04T09:54:18.0656744Z Compilation failures 0 2025-12-04T09:54:18.0657114Z Non-cacheable compilations 0 2025-12-04T09:54:18.0657488Z Non-cacheable calls 25 2025-12-04T09:54:18.0657861Z Non-compilation calls 251 2025-12-04T09:54:18.0658231Z Unsupported compiler calls 0 2025-12-04T09:54:18.0658616Z Average cache write 0.000 s 2025-12-04T09:54:18.0659005Z Average compiler 0.000 s 2025-12-04T09:54:18.0659495Z Average cache read hit 0.000 s 2025-12-04T09:54:18.0659881Z Failed distributed compilations 0 2025-12-04T09:54:18.0660149Z 2025-12-04T09:54:18.0660264Z Non-cacheable reasons: 2025-12-04T09:54:18.0660806Z -E 25 2025-12-04T09:54:18.0661051Z 2025-12-04T09:54:18.0661325Z Cache location s3, name: ossci-compiler-cache-circleci-v2, prefix: / 2025-12-04T09:54:18.0661856Z Version (client) 0.10.0 2025-12-04T09:54:18.0662272Z + sccache --stop-server 2025-12-04T09:54:18.0670179Z Stopping sccache server... 2025-12-04T09:54:18.0672237Z Compile requests 276 2025-12-04T09:54:18.0672644Z Compile requests executed 0 2025-12-04T09:54:18.0673022Z Cache hits 0 2025-12-04T09:54:18.0673444Z Cache misses 0 2025-12-04T09:54:18.0673815Z Cache hits rate - 2025-12-04T09:54:18.0674184Z Cache timeouts 0 2025-12-04T09:54:18.0674535Z Cache read errors 0 2025-12-04T09:54:18.0674901Z Forced recaches 0 2025-12-04T09:54:18.0675376Z Cache write errors 0 2025-12-04T09:54:18.0675912Z Cache errors 0 2025-12-04T09:54:18.0676339Z Compilations 0 2025-12-04T09:54:18.0676720Z Compilation failures 0 2025-12-04T09:54:18.0677108Z Non-cacheable compilations 0 2025-12-04T09:54:18.0677480Z Non-cacheable calls 25 2025-12-04T09:54:18.0677855Z Non-compilation calls 251 2025-12-04T09:54:18.0678236Z Unsupported compiler calls 0 2025-12-04T09:54:18.0678606Z Average cache write 0.000 s 2025-12-04T09:54:18.0679119Z Average compiler 0.000 s 2025-12-04T09:54:18.0679492Z Average cache read hit 0.000 s 2025-12-04T09:54:18.0679862Z Failed distributed compilations 0 2025-12-04T09:54:18.0680127Z 2025-12-04T09:54:18.0680241Z Non-cacheable reasons: 2025-12-04T09:54:18.0680533Z -E 25 2025-12-04T09:54:18.0680765Z 2025-12-04T09:54:18.0681041Z Cache location s3, name: ossci-compiler-cache-circleci-v2, prefix: / 2025-12-04T09:54:18.0681559Z Version (client) 0.10.0 2025-12-04T09:54:18.0681944Z + echo ::endgroup:: 2025-12-04T09:54:18.0682470Z ##[endgroup] 2025-12-04T09:54:18.0682726Z + cleanup_workspace 2025-12-04T09:54:18.0683287Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2025-12-04T09:54:18.0684200Z sudo may print the following warning message that can be ignored. The chown command will still run. 2025-12-04T09:54:18.0684932Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2025-12-04T09:54:18.0685473Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2025-12-04T09:54:18.0686083Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2025-12-04T09:54:18.0686776Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2025-12-04T09:54:18.0687321Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2025-12-04T09:54:18.8308851Z ##[group]Run pytorch/test-infra/.github/actions/upload-benchmark-results@main 2025-12-04T09:54:18.8309443Z with: 2025-12-04T09:54:18.8309731Z benchmark-results-dir: test/test-reports 2025-12-04T09:54:18.8310102Z dry-run: false 2025-12-04T09:54:18.8310376Z schema-version: v3 2025-12-04T09:54:18.8310897Z github-token: *** 2025-12-04T09:54:18.8311167Z env: 2025-12-04T09:54:18.8311405Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:18.8311726Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:18.8312257Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:18.8312839Z ##[endgroup] 2025-12-04T09:54:18.8332907Z ##[group]Run set -eux 2025-12-04T09:54:18.8337962Z set -eux 2025-12-04T09:54:18.8338232Z  2025-12-04T09:54:18.8338478Z if [[ -n "" ]]; then 2025-12-04T09:54:18.8338800Z  source "" 2025-12-04T09:54:18.8339081Z fi 2025-12-04T09:54:18.8339538Z python3 -mpip install boto3==1.35.33 psutil==7.0.0 pynvml==12.0.0 2025-12-04T09:54:18.8340260Z  2025-12-04T09:54:18.8340520Z DEVICE_NAME="" 2025-12-04T09:54:18.8340833Z DEVICE_TYPE="" 2025-12-04T09:54:18.8341128Z  2025-12-04T09:54:18.8341390Z if command -v nvidia-smi; then 2025-12-04T09:54:18.8341935Z  # NB: I'm using PyTorch here to get the device name, however, it needs to 2025-12-04T09:54:18.8342635Z  # install the correct version of PyTorch manually for now. Any PyTorch 2025-12-04T09:54:18.8343272Z  # version is fine, I just use 2.7.1 to satify PYPIDEP linter 2025-12-04T09:54:18.8343783Z  python3 -mpip install torch==2.7.1 2025-12-04T09:54:18.8344199Z elif command -v rocminfo; then 2025-12-04T09:54:18.8344731Z  # NB: Installing torch on ROCm runner with pip here causes CI to fail 2025-12-04T09:54:18.8345393Z  # with a memoryview is too large error only on MI300 runners. Is pip 2025-12-04T09:54:18.8346074Z  # version on ROCm runner there too old? As a workaround, let's use the 2025-12-04T09:54:18.8346769Z  # GPU device name coming from rocminfo instead 2025-12-04T09:54:18.8347194Z  DEVICE_NAME=rocm 2025-12-04T09:54:18.8347771Z  DEVICE_TYPE=$(rocminfo | grep "Marketing Name" | tail -n1 | awk -F':' '{print $2}' | xargs) 2025-12-04T09:54:18.8348373Z fi 2025-12-04T09:54:18.8348622Z  2025-12-04T09:54:18.8348927Z echo "DEVICE_NAME=$DEVICE_NAME" >> $GITHUB_ENV 2025-12-04T09:54:18.8349417Z echo "DEVICE_TYPE=$DEVICE_TYPE" >> $GITHUB_ENV 2025-12-04T09:54:18.8359715Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:18.8360139Z env: 2025-12-04T09:54:18.8360390Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:18.8360702Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:18.8361208Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:18.8361793Z ##[endgroup] 2025-12-04T09:54:18.8400078Z + [[ -n '' ]] 2025-12-04T09:54:18.8400622Z + python3 -mpip install boto3==1.35.33 psutil==7.0.0 pynvml==12.0.0 2025-12-04T09:54:19.1228909Z Defaulting to user installation because normal site-packages is not writeable 2025-12-04T09:54:20.5221594Z Collecting boto3==1.35.33 2025-12-04T09:54:20.5411800Z Downloading boto3-1.35.33-py3-none-any.whl (139 kB) 2025-12-04T09:54:20.9403053Z Collecting psutil==7.0.0 2025-12-04T09:54:20.9455025Z Downloading psutil-7.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (277 kB) 2025-12-04T09:54:20.9797998Z Collecting pynvml==12.0.0 2025-12-04T09:54:20.9840451Z Downloading pynvml-12.0.0-py3-none-any.whl (26 kB) 2025-12-04T09:54:20.9931594Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.35.33) (0.10.0) 2025-12-04T09:54:22.4110576Z Collecting botocore<1.36.0,>=1.35.33 2025-12-04T09:54:22.4165038Z Downloading botocore-1.35.99-py3-none-any.whl (13.3 MB) 2025-12-04T09:54:22.6684683Z Collecting s3transfer<0.11.0,>=0.10.0 2025-12-04T09:54:22.6726396Z Downloading s3transfer-0.10.4-py3-none-any.whl (83 kB) 2025-12-04T09:54:22.7317010Z Collecting nvidia-ml-py<13.0.0a0,>=12.0.0 2025-12-04T09:54:22.7358252Z Downloading nvidia_ml_py-12.575.51-py3-none-any.whl (47 kB) 2025-12-04T09:54:22.7472722Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (2.8.1) 2025-12-04T09:54:22.7483868Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.25.10) 2025-12-04T09:54:22.9263584Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.15.0) 2025-12-04T09:54:23.0833624Z Installing collected packages: botocore, s3transfer, nvidia-ml-py, pynvml, psutil, boto3 2025-12-04T09:54:23.7097995Z Attempting uninstall: nvidia-ml-py 2025-12-04T09:54:23.7098870Z Found existing installation: nvidia-ml-py 11.525.84 2025-12-04T09:54:23.7114862Z Uninstalling nvidia-ml-py-11.525.84: 2025-12-04T09:54:23.7339259Z Successfully uninstalled nvidia-ml-py-11.525.84 2025-12-04T09:54:23.8065949Z Attempting uninstall: psutil 2025-12-04T09:54:23.8066402Z Found existing installation: psutil 5.9.8 2025-12-04T09:54:23.8150248Z Uninstalling psutil-5.9.8: 2025-12-04T09:54:23.8156903Z Successfully uninstalled psutil-5.9.8 2025-12-04T09:54:24.0169915Z Successfully installed boto3-1.35.33 botocore-1.35.99 nvidia-ml-py-12.575.51 psutil-7.0.0 pynvml-12.0.0 s3transfer-0.10.4 2025-12-04T09:54:24.1340512Z + DEVICE_NAME= 2025-12-04T09:54:24.1340832Z + DEVICE_TYPE= 2025-12-04T09:54:24.1341121Z + command -v nvidia-smi 2025-12-04T09:54:24.1341430Z + command -v rocminfo 2025-12-04T09:54:24.1341703Z + echo DEVICE_NAME= 2025-12-04T09:54:24.1341979Z + echo DEVICE_TYPE= 2025-12-04T09:54:24.1367143Z ##[group]Run set -eux 2025-12-04T09:54:24.1367443Z set -eux 2025-12-04T09:54:24.1367722Z  2025-12-04T09:54:24.1367999Z if [[ -z "${GITHUB_TOKEN}" ]]; then 2025-12-04T09:54:24.1368398Z  echo "Missing github-token input" 2025-12-04T09:54:24.1368771Z  exit 1 2025-12-04T09:54:24.1369027Z fi 2025-12-04T09:54:24.1376300Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:24.1376739Z env: 2025-12-04T09:54:24.1376994Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:24.1377293Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:24.1377817Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:24.1378400Z DEVICE_NAME: 2025-12-04T09:54:24.1378656Z DEVICE_TYPE: 2025-12-04T09:54:24.1379137Z GITHUB_TOKEN: *** 2025-12-04T09:54:24.1379406Z ##[endgroup] 2025-12-04T09:54:24.1413257Z + [[ -z *** ]] 2025-12-04T09:54:24.1474779Z ##[group]Run pytorch/test-infra/.github/actions/get-workflow-job-id@main 2025-12-04T09:54:24.1475298Z with: 2025-12-04T09:54:24.1475736Z github-token: *** 2025-12-04T09:54:24.1475993Z env: 2025-12-04T09:54:24.1476242Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:24.1476564Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:24.1477082Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:24.1477678Z DEVICE_NAME: 2025-12-04T09:54:24.1477942Z DEVICE_TYPE: 2025-12-04T09:54:24.1478206Z ##[endgroup] 2025-12-04T09:54:24.1494234Z ##[group]Run set -eux 2025-12-04T09:54:24.1494522Z set -eux 2025-12-04T09:54:24.1494815Z  2025-12-04T09:54:24.1495370Z python3 "${GITHUB_ACTION_PATH}/../../scripts/get_workflow_job_id.py" "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2025-12-04T09:54:24.1502911Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:24.1503361Z env: 2025-12-04T09:54:24.1503615Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:24.1503941Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:24.1504610Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:24.1505221Z DEVICE_NAME: 2025-12-04T09:54:24.1505485Z DEVICE_TYPE: 2025-12-04T09:54:24.1505976Z GITHUB_TOKEN: *** 2025-12-04T09:54:24.1506255Z ##[endgroup] 2025-12-04T09:54:24.1532744Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/get-workflow-job-id/../../scripts/get_workflow_job_id.py 19923066595 i-01ba3f41dbfc68ae5 2025-12-04T09:54:26.3019710Z setting job-id=57118563305 2025-12-04T09:54:26.3020586Z setting job-name=periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:54:26.3145345Z ##[group]Run set -eux 2025-12-04T09:54:26.3145670Z set -eux 2025-12-04T09:54:26.3145921Z  2025-12-04T09:54:26.3146177Z if [[ -n "" ]]; then 2025-12-04T09:54:26.3146500Z  source "" 2025-12-04T09:54:26.3146870Z fi 2025-12-04T09:54:26.3147148Z  2025-12-04T09:54:26.3147607Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_metadata.py" \ 2025-12-04T09:54:26.3148393Z  --schema-version "${SCHEMA_VERSION}" \ 2025-12-04T09:54:26.3148792Z  --repo "${REPO}" \ 2025-12-04T09:54:26.3149145Z  --head-branch "${HEAD_BRANCH}" \ 2025-12-04T09:54:26.3149543Z  --head-sha "${HEAD_SHA}" \ 2025-12-04T09:54:26.3149926Z  --workflow-id "${WORKFLOW_RUN_ID}" \ 2025-12-04T09:54:26.3150349Z  --run-attempt "${RUN_ATTEMPT}" \ 2025-12-04T09:54:26.3150736Z  --job-id "${JOB_ID}" \ 2025-12-04T09:54:26.3151091Z  --job-name "${JOB_NAME}" 2025-12-04T09:54:26.3157600Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:26.3158060Z env: 2025-12-04T09:54:26.3158316Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:26.3158729Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:26.3159258Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:26.3159834Z DEVICE_NAME: 2025-12-04T09:54:26.3160107Z DEVICE_TYPE: 2025-12-04T09:54:26.3160353Z SCHEMA_VERSION: v3 2025-12-04T09:54:26.3160641Z REPO: pytorch/pytorch 2025-12-04T09:54:26.3160941Z HEAD_BRANCH: refs/heads/main 2025-12-04T09:54:26.3161303Z HEAD_SHA: ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 2025-12-04T09:54:26.3161708Z WORKFLOW_RUN_ID: 19923066595 2025-12-04T09:54:26.3162019Z RUN_ATTEMPT: 1 2025-12-04T09:54:26.3162263Z JOB_ID: 57118563305 2025-12-04T09:54:26.3162933Z JOB_NAME: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:54:26.3163676Z ##[endgroup] 2025-12-04T09:54:26.3190790Z + [[ -n '' ]] 2025-12-04T09:54:26.3193663Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_metadata.py --schema-version v3 --repo pytorch/pytorch --head-branch refs/heads/main --head-sha ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32 --workflow-id 19923066595 --run-attempt 1 --job-id 57118563305 --job-name 'periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2)' 2025-12-04T09:54:26.3568152Z ##[group]Run set -eux 2025-12-04T09:54:26.3568463Z set -eux 2025-12-04T09:54:26.3568721Z  2025-12-04T09:54:26.3568955Z if [[ -n "" ]]; then 2025-12-04T09:54:26.3569267Z  source "" 2025-12-04T09:54:26.3569796Z fi 2025-12-04T09:54:26.3570028Z  2025-12-04T09:54:26.3570569Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_runners_info.py" 2025-12-04T09:54:26.3577634Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:26.3578086Z env: 2025-12-04T09:54:26.3578331Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:26.3578649Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:26.3579343Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:26.3579930Z DEVICE_NAME: 2025-12-04T09:54:26.3580209Z DEVICE_TYPE: 2025-12-04T09:54:26.3580475Z ##[endgroup] 2025-12-04T09:54:26.3607066Z + [[ -n '' ]] 2025-12-04T09:54:26.4092765Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_runners_info.py 2025-12-04T09:54:26.4093894Z INFO:root:Fail to import torch to get the device name 2025-12-04T09:54:26.4203300Z ##[group]Run set -eux 2025-12-04T09:54:26.4203601Z set -eux 2025-12-04T09:54:26.4203869Z  2025-12-04T09:54:26.4204166Z # TODO (huydhn): Implement this part 2025-12-04T09:54:26.4204614Z echo "dependencies={}" >> "${GITHUB_OUTPUT}" 2025-12-04T09:54:26.4210953Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:26.4211402Z env: 2025-12-04T09:54:26.4211660Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:26.4211963Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:26.4212510Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:26.4213364Z DEVICE_NAME: 2025-12-04T09:54:26.4213620Z DEVICE_TYPE: 2025-12-04T09:54:26.4213859Z ##[endgroup] 2025-12-04T09:54:26.4240958Z + echo 'dependencies={}' 2025-12-04T09:54:26.4278085Z ##[group]Run set -eux 2025-12-04T09:54:26.4278525Z set -eux 2025-12-04T09:54:26.4278781Z  2025-12-04T09:54:26.4279028Z if [[ -n "" ]]; then 2025-12-04T09:54:26.4279323Z  source "" 2025-12-04T09:54:26.4279593Z fi 2025-12-04T09:54:26.4279836Z  2025-12-04T09:54:26.4280122Z if [[ ! -d "${BENCHMARK_RESULTS_DIR}" ]]; then 2025-12-04T09:54:26.4280626Z  echo "${BENCHMARK_RESULTS_DIR} does not exist, skipping" 2025-12-04T09:54:26.4281192Z  # We don't want the job to fail if the directory doesn't exist 2025-12-04T09:54:26.4281640Z  exit 0 2025-12-04T09:54:26.4281894Z fi 2025-12-04T09:54:26.4282145Z  2025-12-04T09:54:26.4282414Z if [[ "${DRY_RUN}" == "true" ]]; then 2025-12-04T09:54:26.4282966Z  python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \ 2025-12-04T09:54:26.4283612Z  --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \ 2025-12-04T09:54:26.4284109Z  --metadata "${BENCHMARK_METADATA}" \ 2025-12-04T09:54:26.4284503Z  --runners "${RUNNER_INFO}" \ 2025-12-04T09:54:26.4284901Z  --dependencies "${DEPENDENCIES}" \ 2025-12-04T09:54:26.4285282Z  --dry-run 2025-12-04T09:54:26.4285560Z else 2025-12-04T09:54:26.4286014Z  python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \ 2025-12-04T09:54:26.4286645Z  --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \ 2025-12-04T09:54:26.4287137Z  --metadata "${BENCHMARK_METADATA}" \ 2025-12-04T09:54:26.4287550Z  --runners "${RUNNER_INFO}" \ 2025-12-04T09:54:26.4287952Z  --dependencies "${DEPENDENCIES}" 2025-12-04T09:54:26.4288313Z fi 2025-12-04T09:54:26.4294040Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:26.4294476Z env: 2025-12-04T09:54:26.4294712Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:26.4295027Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:26.4295551Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:26.4296122Z DEVICE_NAME: 2025-12-04T09:54:26.4296380Z DEVICE_TYPE: 2025-12-04T09:54:26.4296676Z BENCHMARK_RESULTS_DIR: test/test-reports 2025-12-04T09:54:26.4297028Z DRY_RUN: false 2025-12-04T09:54:26.4298921Z BENCHMARK_METADATA: {"timestamp": 1764842066, "schema_version": "v3", "name": "periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2)", "repo": "pytorch/pytorch", "head_branch": "refs/heads/main", "head_sha": "ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32", "workflow_id": 19923066595, "run_attempt": 1, "job_id": 57118563305} 2025-12-04T09:54:26.4301794Z RUNNER_INFO: [{"cpu_info": "x86_64", "cpu_count": 40, "avail_mem_in_gb": 157, "extra_info": {"hostname": "ip-10-0-76-7.ec2.internal"}, "name": "", "type": ""}] 2025-12-04T09:54:26.4302580Z DEPENDENCIES: {} 2025-12-04T09:54:26.4302863Z ##[endgroup] 2025-12-04T09:54:26.4333078Z + [[ -n '' ]] 2025-12-04T09:54:26.4333414Z + [[ ! -d test/test-reports ]] 2025-12-04T09:54:26.4333774Z + [[ false == \t\r\u\e ]] 2025-12-04T09:54:26.4337439Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py --benchmark-results-dir test/test-reports --metadata '{"timestamp": 1764842066, "schema_version": "v3", "name": "periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2)", "repo": "pytorch/pytorch", "head_branch": "refs/heads/main", "head_sha": "ffd9b0fb4355e97af82fc42cf185c3ffa0fc0a32", "workflow_id": 19923066595, "run_attempt": 1, "job_id": 57118563305}' --runners '[{"cpu_info": "x86_64", "cpu_count": 40, "avail_mem_in_gb": 157, "extra_info": {"hostname": "ip-10-0-76-7.ec2.internal"}, "name": "", "type": ""}]' --dependencies '{}' 2025-12-04T09:54:26.6048785Z INFO:root:Upload test/test-reports/inference_huggingface.json to s3://ossci-benchmarks/v3/pytorch/pytorch/19923066595/57118563305/inference_huggingface.json 2025-12-04T09:54:26.6451681Z INFO:botocore.credentials:Found credentials from IAM Role: gh-ci-github-action-runners-runner-role 2025-12-04T09:54:26.8798093Z ##[group]Run cat test/**/*_toprint.log || true 2025-12-04T09:54:26.8798587Z cat test/**/*_toprint.log || true 2025-12-04T09:54:26.8806224Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:26.8806669Z env: 2025-12-04T09:54:26.8806930Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:26.8807249Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:26.8807772Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:26.8808391Z DEVICE_NAME: 2025-12-04T09:54:26.8808657Z DEVICE_TYPE: 2025-12-04T09:54:26.8808919Z ##[endgroup] 2025-12-04T09:54:26.8911618Z cat: 'test/**/*_toprint.log': No such file or directory 2025-12-04T09:54:26.8941030Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2025-12-04T09:54:26.8941445Z kill "$MONITOR_SCRIPT_PID" 2025-12-04T09:54:26.8947966Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:26.8948406Z env: 2025-12-04T09:54:26.8948662Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:26.8948981Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:26.8949500Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:26.8950100Z DEVICE_NAME: 2025-12-04T09:54:26.8950369Z DEVICE_TYPE: 2025-12-04T09:54:26.8950649Z MONITOR_SCRIPT_PID: 57732 2025-12-04T09:54:26.8950951Z ##[endgroup] 2025-12-04T09:54:26.8976028Z /home/ec2-user/actions-runner/_work/_temp/3c11d523-a544-4255-9020-e94684c033bb.sh: line 1: kill: (57732) - No such process 2025-12-04T09:54:26.8988334Z ##[error]Process completed with exit code 1. 2025-12-04T09:54:26.9156946Z Prepare all required actions 2025-12-04T09:54:26.9157439Z Getting action download info 2025-12-04T09:54:27.0643086Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2025-12-04T09:54:27.3255169Z Download action repository 'actions/upload-artifact@v4' (SHA:ea165f8d65b6e75b540449e92b4886f43607fa02) 2025-12-04T09:54:27.8657652Z ##[group]Run ./.github/actions/upload-test-artifacts 2025-12-04T09:54:27.8658102Z with: 2025-12-04T09:54:27.8658612Z file-suffix: test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305 2025-12-04T09:54:27.8659225Z s3-bucket: gha-artifacts 2025-12-04T09:54:27.8659523Z env: 2025-12-04T09:54:27.8659767Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:27.8660060Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:27.8660573Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:27.8661169Z DEVICE_NAME: 2025-12-04T09:54:27.8661467Z DEVICE_TYPE: 2025-12-04T09:54:27.8661711Z ##[endgroup] 2025-12-04T09:54:27.8688635Z ##[group]Run # Remove any previous test jsons if they exist 2025-12-04T09:54:27.8689151Z # Remove any previous test jsons if they exist 2025-12-04T09:54:27.8689593Z rm -f test-jsons-*.zip 2025-12-04T09:54:27.8690091Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test/test-reports -i '*.json' 2025-12-04T09:54:27.8697059Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:27.8697504Z env: 2025-12-04T09:54:27.8697762Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:27.8698083Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:27.8698605Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:27.8699282Z DEVICE_NAME: 2025-12-04T09:54:27.8699553Z DEVICE_TYPE: 2025-12-04T09:54:27.8700070Z FILE_SUFFIX: test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305 2025-12-04T09:54:27.8701331Z ##[endgroup] 2025-12-04T09:54:27.8863183Z adding: test/test-reports/inference_huggingface.json (deflated 99%) 2025-12-04T09:54:27.8894718Z ##[group]Run # Remove any previous test reports if they exist 2025-12-04T09:54:27.8895249Z # Remove any previous test reports if they exist 2025-12-04T09:54:27.8895694Z rm -f test-reports-*.zip 2025-12-04T09:54:27.8896243Z zip -r "test-reports-${FILE_SUFFIX}.zip" test/test-reports -i '*.xml' -i '*.csv' 2025-12-04T09:54:27.8903191Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:27.8903822Z env: 2025-12-04T09:54:27.8904223Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:27.8904635Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:27.8905166Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:27.8905840Z DEVICE_NAME: 2025-12-04T09:54:27.8906113Z DEVICE_TYPE: 2025-12-04T09:54:27.8906810Z FILE_SUFFIX: test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305 2025-12-04T09:54:27.8907483Z ##[endgroup] 2025-12-04T09:54:27.8961758Z adding: test/test-reports/inference_huggingface.csv (deflated 62%) 2025-12-04T09:54:27.8962547Z adding: test/test-reports/inference_huggingface_graph_breaks.csv (deflated 85%) 2025-12-04T09:54:27.8963356Z adding: test/test-reports/inference_huggingface_graph_break_deduped.csv (deflated 65%) 2025-12-04T09:54:27.8987278Z ##[group]Run # Remove any previous usage logs if they exist 2025-12-04T09:54:27.8987879Z # Remove any previous usage logs if they exist 2025-12-04T09:54:27.8988980Z rm -f logs-*.zip 2025-12-04T09:54:27.8989399Z zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' || true 2025-12-04T09:54:27.8990018Z zip -r "logs-${FILE_SUFFIX}.zip" test/test-reports -i '*.log' || true 2025-12-04T09:54:27.8996169Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:27.8996621Z env: 2025-12-04T09:54:27.8996878Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:27.8997200Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:27.8997897Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:27.8998502Z DEVICE_NAME: 2025-12-04T09:54:27.8998772Z DEVICE_TYPE: 2025-12-04T09:54:27.8999393Z FILE_SUFFIX: test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305 2025-12-04T09:54:27.8999999Z ##[endgroup] 2025-12-04T09:54:27.9079143Z adding: usage_log.txt (deflated 58%) 2025-12-04T09:54:27.9082054Z 2025-12-04T09:54:27.9082546Z zip error: Nothing to do! (logs-test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305.zip) 2025-12-04T09:54:27.9203660Z ##[group]Run # Remove any previous debugging artifacts if they exist 2025-12-04T09:54:27.9204292Z # Remove any previous debugging artifacts if they exist 2025-12-04T09:54:27.9204767Z rm -f debug-*.zip 2025-12-04T09:54:27.9205088Z if [ -d 'test/debug' ]; then 2025-12-04T09:54:27.9205520Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2025-12-04T09:54:27.9205923Z fi 2025-12-04T09:54:27.9211667Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:27.9212104Z env: 2025-12-04T09:54:27.9212362Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:27.9212667Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:27.9213307Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:27.9213889Z DEVICE_NAME: 2025-12-04T09:54:27.9214135Z DEVICE_TYPE: 2025-12-04T09:54:27.9214645Z FILE_SUFFIX: test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305 2025-12-04T09:54:27.9215241Z ##[endgroup] 2025-12-04T09:54:27.9301579Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-12-04T09:54:27.9301976Z with: 2025-12-04T09:54:27.9302227Z s3-bucket: gha-artifacts 2025-12-04T09:54:27.9302597Z s3-prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:54:27.9302994Z retention-days: 14 2025-12-04T09:54:27.9303444Z if-no-files-found: warn 2025-12-04T09:54:27.9303764Z path: test-jsons-*.zip 2025-12-04T09:54:27.9304053Z name: artifact 2025-12-04T09:54:27.9304300Z region: us-east-1 2025-12-04T09:54:27.9304558Z env: 2025-12-04T09:54:27.9304790Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:27.9305086Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:27.9305604Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:27.9306188Z DEVICE_NAME: 2025-12-04T09:54:27.9306438Z DEVICE_TYPE: 2025-12-04T09:54:27.9306829Z ##[endgroup] 2025-12-04T09:54:28.3383372Z NOTE: s3-prefix specified, ignoring name parameter 2025-12-04T09:54:28.3383944Z With the provided path, there will be 1 file uploaded 2025-12-04T09:54:28.3384490Z Uploading to s3 prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:54:28.3439794Z Starting upload of test-jsons-test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305.zip 2025-12-04T09:54:28.4802994Z Finished upload of test-jsons-test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305.zip 2025-12-04T09:54:28.4984320Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-12-04T09:54:28.4984693Z with: 2025-12-04T09:54:28.4984949Z s3-bucket: gha-artifacts 2025-12-04T09:54:28.4985320Z s3-prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:54:28.4985713Z retention-days: 14 2025-12-04T09:54:28.4986006Z if-no-files-found: error 2025-12-04T09:54:28.4986323Z path: test-reports-*.zip 2025-12-04T09:54:28.4986615Z name: artifact 2025-12-04T09:54:28.4987032Z region: us-east-1 2025-12-04T09:54:28.4987466Z env: 2025-12-04T09:54:28.4987704Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:28.4988045Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:28.4988583Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:28.4989182Z DEVICE_NAME: 2025-12-04T09:54:28.4989438Z DEVICE_TYPE: 2025-12-04T09:54:28.4989705Z ##[endgroup] 2025-12-04T09:54:28.9139643Z NOTE: s3-prefix specified, ignoring name parameter 2025-12-04T09:54:28.9140567Z With the provided path, there will be 1 file uploaded 2025-12-04T09:54:28.9141115Z Uploading to s3 prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:54:28.9194266Z Starting upload of test-reports-test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305.zip 2025-12-04T09:54:29.0496539Z Finished upload of test-reports-test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305.zip 2025-12-04T09:54:29.0686790Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-12-04T09:54:29.0687169Z with: 2025-12-04T09:54:29.0687427Z s3-bucket: gha-artifacts 2025-12-04T09:54:29.0687796Z s3-prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:54:29.0688194Z retention-days: 14 2025-12-04T09:54:29.0688490Z if-no-files-found: ignore 2025-12-04T09:54:29.0688806Z path: logs-*.zip 2025-12-04T09:54:29.0689061Z name: artifact 2025-12-04T09:54:29.0689320Z region: us-east-1 2025-12-04T09:54:29.0689599Z env: 2025-12-04T09:54:29.0689869Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:29.0690178Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:29.0690700Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:29.0698611Z DEVICE_NAME: 2025-12-04T09:54:29.0698910Z DEVICE_TYPE: 2025-12-04T09:54:29.0699178Z ##[endgroup] 2025-12-04T09:54:29.4539271Z NOTE: s3-prefix specified, ignoring name parameter 2025-12-04T09:54:29.4539921Z With the provided path, there will be 1 file uploaded 2025-12-04T09:54:29.4540455Z Uploading to s3 prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:54:29.4598361Z Starting upload of logs-test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305.zip 2025-12-04T09:54:29.5928998Z Finished upload of logs-test-cpu_inductor_freezing_avx2_huggingface-1-1-linux.10xlarge.avx2_57118563305.zip 2025-12-04T09:54:29.6117416Z ##[group]Run seemethere/upload-artifact-s3@v5 2025-12-04T09:54:29.6117809Z with: 2025-12-04T09:54:29.6118243Z s3-bucket: gha-artifacts 2025-12-04T09:54:29.6118740Z s3-prefix: pytorch/pytorch/19923066595/1/artifact 2025-12-04T09:54:29.6119136Z retention-days: 14 2025-12-04T09:54:29.6119433Z if-no-files-found: ignore 2025-12-04T09:54:29.6119747Z path: debug-*.zip 2025-12-04T09:54:29.6120001Z name: artifact 2025-12-04T09:54:29.6120264Z region: us-east-1 2025-12-04T09:54:29.6120524Z env: 2025-12-04T09:54:29.6120751Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:29.6121058Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:29.6121578Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:29.6122158Z DEVICE_NAME: 2025-12-04T09:54:29.6122402Z DEVICE_TYPE: 2025-12-04T09:54:29.6122658Z ##[endgroup] 2025-12-04T09:54:29.9898405Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2025-12-04T09:54:30.0135918Z ##[group]Run # shellcheck disable=SC2156 2025-12-04T09:54:30.0136365Z # shellcheck disable=SC2156 2025-12-04T09:54:30.0137098Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2025-12-04T09:54:30.0144055Z shell: /usr/bin/bash -e {0} 2025-12-04T09:54:30.0144374Z env: 2025-12-04T09:54:30.0144630Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:30.0144932Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:30.0145461Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:30.0146046Z DEVICE_NAME: 2025-12-04T09:54:30.0146308Z DEVICE_TYPE: 2025-12-04T09:54:30.0146555Z ##[endgroup] 2025-12-04T09:54:30.3875422Z Prepare all required actions 2025-12-04T09:54:30.3875887Z Getting action download info 2025-12-04T09:54:30.5357780Z Download action repository 'actions/setup-python@v6' (SHA:83679a892e2d95755f2dac6acb0bfd1e9ac5d548) 2025-12-04T09:54:30.9119509Z ##[group]Run ./.github/actions/upload-utilization-stats 2025-12-04T09:54:30.9119943Z with: 2025-12-04T09:54:30.9120191Z job_id: 57118563305 2025-12-04T09:54:30.9120902Z job_name: periodic-dynamo-benchmarks-cpu-test / test (cpu_inductor_freezing_avx2_huggingface, 1, 1, linux.10xlarge.avx2) 2025-12-04T09:54:30.9121666Z workflow_name: inductor-periodic 2025-12-04T09:54:30.9122028Z workflow_run_id: 19923066595 2025-12-04T09:54:30.9122358Z workflow_attempt: 1 2025-12-04T09:54:30.9122624Z env: 2025-12-04T09:54:30.9122873Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:30.9123184Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:30.9123690Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:30.9124325Z DEVICE_NAME: 2025-12-04T09:54:30.9124574Z DEVICE_TYPE: 2025-12-04T09:54:30.9124828Z ##[endgroup] 2025-12-04T09:54:30.9168361Z ##[group]Run actions/setup-python@v6 2025-12-04T09:54:30.9168723Z with: 2025-12-04T09:54:30.9168979Z python-version: 3.10 2025-12-04T09:54:30.9169267Z check-latest: false 2025-12-04T09:54:30.9169668Z token: *** 2025-12-04T09:54:30.9169939Z update-environment: true 2025-12-04T09:54:30.9170278Z allow-prereleases: false 2025-12-04T09:54:30.9170573Z freethreaded: false 2025-12-04T09:54:30.9170848Z env: 2025-12-04T09:54:30.9171090Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:30.9171379Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:30.9171896Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:30.9172472Z DEVICE_NAME: 2025-12-04T09:54:30.9172718Z DEVICE_TYPE: 2025-12-04T09:54:30.9172969Z ##[endgroup] 2025-12-04T09:54:31.0812577Z ##[group]Installed versions 2025-12-04T09:54:31.0828022Z Version 3.10 was not found in the local cache 2025-12-04T09:54:31.1036347Z (node:78724) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. 2025-12-04T09:54:31.1037307Z (Use `node --trace-deprecation ...` to show where the warning was created) 2025-12-04T09:54:31.4740788Z ##[error]The version '3.10' with architecture 'x64' was not found for this operating system. The list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json 2025-12-04T09:54:31.4919088Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2025-12-04T09:54:31.4919598Z with: 2025-12-04T09:54:31.4919816Z env: 2025-12-04T09:54:31.4920060Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:31.4920372Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:31.4920878Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:31.4921458Z DEVICE_NAME: 2025-12-04T09:54:31.4921716Z DEVICE_TYPE: 2025-12-04T09:54:31.4921958Z ##[endgroup] 2025-12-04T09:54:31.4944576Z ##[group]Run set -eou pipefail 2025-12-04T09:54:31.4944941Z set -eou pipefail 2025-12-04T09:54:31.4945261Z  2025-12-04T09:54:31.4945697Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2025-12-04T09:54:31.4946257Z for _ in $(seq 1440); do 2025-12-04T09:54:31.4946752Z  # Break if no ssh session exists anymore 2025-12-04T09:54:31.4947197Z  if [ "$(who)" = "" ]; then 2025-12-04T09:54:31.4947588Z  break 2025-12-04T09:54:31.4947868Z  fi 2025-12-04T09:54:31.4948137Z  echo "." 2025-12-04T09:54:31.4948423Z  sleep 5 2025-12-04T09:54:31.4948687Z done 2025-12-04T09:54:31.4955088Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:31.4955543Z env: 2025-12-04T09:54:31.4955788Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:31.4956107Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:31.4956641Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:31.4957226Z DEVICE_NAME: 2025-12-04T09:54:31.4957489Z DEVICE_TYPE: 2025-12-04T09:54:31.4957750Z ##[endgroup] 2025-12-04T09:54:31.4993424Z Holding runner for 2 hours until all ssh sessions have logged out 2025-12-04T09:54:31.5112698Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2025-12-04T09:54:31.5113376Z # ignore expansion of "docker ps -q" since it could be empty 2025-12-04T09:54:31.5113886Z # shellcheck disable=SC2046 2025-12-04T09:54:31.5114279Z docker stop $(docker ps -q) || true 2025-12-04T09:54:31.5114683Z # Prune all of the docker images 2025-12-04T09:54:31.5115059Z docker system prune -af 2025-12-04T09:54:31.5121236Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:31.5121654Z env: 2025-12-04T09:54:31.5121885Z GIT_DEFAULT_BRANCH: main 2025-12-04T09:54:31.5122169Z HAS_NVIDIA_GPU: false 2025-12-04T09:54:31.5122663Z DOCKER_CONTAINER_ID: 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:31.5123220Z DEVICE_NAME: 2025-12-04T09:54:31.5123453Z DEVICE_TYPE: 2025-12-04T09:54:31.5123694Z ##[endgroup] 2025-12-04T09:54:42.1572446Z 77ce1fdc01d3 2025-12-04T09:54:42.6657809Z Deleted Containers: 2025-12-04T09:54:42.6658313Z 77ce1fdc01d35ab7cd11d391459ab79a5675e0353bb9e106925d4773f0ce1ebe 2025-12-04T09:54:42.6658777Z 2025-12-04T09:54:55.3136604Z Deleted Images: 2025-12-04T09:54:55.3137758Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-py3-gcc11-inductor-benchmarks-f0cd68561080d537ef3d3d6f81b25a6416ad600a 2025-12-04T09:54:55.3139332Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image@sha256:b178ee928adfcab963ebbc6ea05ea1a0f0c605bb095e9076f010bf92d150869d 2025-12-04T09:54:55.3140411Z deleted: sha256:40bd34edc3497e05e2150804a15313235b6426bcc4b8c19c03c3daf5524be913 2025-12-04T09:54:55.3141159Z deleted: sha256:947006dce32e090f6b67d556f519398b957121651577f10819c7b179a5b6b938 2025-12-04T09:54:55.3141903Z deleted: sha256:dca16630fc5508e008012b760ac178fac5af74cb679706a3d9898100501d6f8e 2025-12-04T09:54:55.3142645Z deleted: sha256:a35e2e4cfaaa05e1f193fbe67c4151ded52737cb30c75429050cb7cd7777da23 2025-12-04T09:54:55.3143409Z deleted: sha256:d91e1a0a76e64abd7933fc58b8a553d76ac6cd82269c1bf701ade4e2c5ab58ff 2025-12-04T09:54:55.3144536Z deleted: sha256:f96fff40164ffa27ae535a4e5da2bbf2b18ec293366adc73519c830606f07baa 2025-12-04T09:54:55.3145457Z deleted: sha256:649089a3156b98c2cbf16b0af2b1ebdaa8644f9bf90534d9a1f7657b78b9f90e 2025-12-04T09:54:55.3146211Z deleted: sha256:aebabb7ea7535ad2aca44b23eb308b16d5de44f5b412574f94620d923436f42e 2025-12-04T09:54:55.3147120Z deleted: sha256:bcb08dc52ae55732fc33ab8c011347049a9279cc5f45a50d960c95d3a7cd5a3d 2025-12-04T09:54:55.3148047Z deleted: sha256:aa538cda5b22ea6e81fb69326637459a1356cce39b618657a075ce25d9124876 2025-12-04T09:54:55.3148877Z deleted: sha256:e28671ddaf02cbac7c1dcfdb2b08a40d3b7e2dab69c9a9e6c48ede3d8d550d66 2025-12-04T09:54:55.3149746Z deleted: sha256:ba797a4f759946d2fa7414073a5590cc12337aaa43197c35326ce456a3f84adf 2025-12-04T09:54:55.3158808Z deleted: sha256:25b28b66e2a53c3498a56aff9f543bf3a4641534c9e69741c673e9d018c652d8 2025-12-04T09:54:55.3159576Z deleted: sha256:860240198a7e7dcec697a58bcf979cfde64334033f2e97d4c9609277e051558c 2025-12-04T09:54:55.3160357Z deleted: sha256:39de54a9c331b8da952419605b26c35862b4cbb8c8a0967cc536a02b96eda874 2025-12-04T09:54:55.3161110Z deleted: sha256:6f78957316e0887707bc6534ea2aa43d1cd016130ec6a0d3148370b879696e1a 2025-12-04T09:54:55.3161991Z deleted: sha256:8656f7aa172ef2a37ec1524ac6f3ecd2b3fd9675a78a777b9a902d0f26047ea9 2025-12-04T09:54:55.3162745Z deleted: sha256:255344ff6c764e507ea2f21fbbdc6e0a1aaab0e5e0835935494c9df7d620e687 2025-12-04T09:54:55.3163487Z deleted: sha256:b6a1addb4370e1500d2fd65255b80922ad308c95a846a3460be730442d75fbe4 2025-12-04T09:54:55.3164226Z deleted: sha256:8dafa1eff3c9aada8b6725626ba0dd7de62719fb5ff307be3005cf2c6e72e2df 2025-12-04T09:54:55.3164994Z deleted: sha256:9e3fde92e8ff4f05cadac548766ef5ef9e8beb3c89cb48b4e2a17843ff60d649 2025-12-04T09:54:55.3165748Z deleted: sha256:7169a6f70d1f624733e52107b4913effe35e1e63c99388164b0afd2345658178 2025-12-04T09:54:55.3166466Z deleted: sha256:4c0d4c67766b690c1f1834310cd3898a900cd9742b7157de7e8f64d651600e35 2025-12-04T09:54:55.3167199Z deleted: sha256:7343a9c5c68976b0328278e691ddbcf26f619f989ca2a51933ac8bf637d1d69c 2025-12-04T09:54:55.3167956Z deleted: sha256:b790f29be6db3408a1a53fabdc69e76bb38cef221474a4f3aef1e233bb85d6d3 2025-12-04T09:54:55.3168701Z deleted: sha256:ed0831c232a6a410214e113695ab9288063623b523a6032a05b443fa3e9a68ae 2025-12-04T09:54:55.3169414Z deleted: sha256:554ed49d988a9f7a1381d193479542175c40d702a22d246ecef7958a8662a2eb 2025-12-04T09:54:55.3170153Z deleted: sha256:47b04d260bfb2b1f68a2f12aa70926ae2a2a53be6f4d8811d6fbe17fd7f13e17 2025-12-04T09:54:55.3170903Z deleted: sha256:6eb59e87109db04da60df057ff6b232045b6df4e2729ce7534a7a77fbe663761 2025-12-04T09:54:55.3171643Z deleted: sha256:0d9763e3d6e83eee92752c3a5dcfce74703dfa0aba42042761d7600edd95baa5 2025-12-04T09:54:55.3172388Z deleted: sha256:5811882cf3f14a50f47dc91a87d1be09daf3c20494aaf7e7343554066e9f90b7 2025-12-04T09:54:55.3173120Z deleted: sha256:4a6f4f545e9062949802a6c597609ef772633eae7d6925d41823d2b1a8991d4d 2025-12-04T09:54:55.3173849Z deleted: sha256:69a716cb9dc95d7d58d1416f4d127e49885c5d96882dc593686a6eb97200582a 2025-12-04T09:54:55.3174577Z deleted: sha256:0542610effe51f46308d4570438af559e4f6a263f38714ea04c78e4661d1909a 2025-12-04T09:54:55.3175309Z deleted: sha256:86dcdb716a38a00a400b483f42e4ac58714a70e89f5671b50ca5c45d8f577217 2025-12-04T09:54:55.3176066Z deleted: sha256:a9aaf6de8156c1b1a8ece816adbc7d2ed6edabb95e36732737ca293b26619b40 2025-12-04T09:54:55.3176833Z deleted: sha256:57ce7fcbaaa86795f8f7e6290c0aaa8622e13c9c82191d34952c0a4d2e203326 2025-12-04T09:54:55.3177579Z deleted: sha256:a90408b5f4c06a3ba881facc34fa55beb4ae4e51823e7f8d933e611f68465f4a 2025-12-04T09:54:55.3178341Z deleted: sha256:1ac1aab20bc98902f03ee9c657ff435af431f40a81ac9cebf1cd91d9041bc6b1 2025-12-04T09:54:55.3179093Z deleted: sha256:89e6c1c76b21fbe98c153a32627fa1b2337a8caa744b09b28253b67cb84012cb 2025-12-04T09:54:55.3179825Z deleted: sha256:3765d58e17aa81539a8e1eb3cb45914b48f8fb53f7ac12c1bf06c9390e274a93 2025-12-04T09:54:55.3180689Z deleted: sha256:6056a51594e8137d3425100cc233c8ab0f09dded2dc491ba083e8d7c0e1f1143 2025-12-04T09:54:55.3182057Z deleted: sha256:43a55512a68fc9eb9c622a73682dbf9f3c83f1b499287d9e12151f559edabe85 2025-12-04T09:54:55.3183459Z deleted: sha256:a5ad8e88a7e08768a61feeb801fb5a4c5041587feb2a2c77c35b285e869c25c9 2025-12-04T09:54:55.3184388Z deleted: sha256:55fd389078a5792f567a3592821e38fcfa0b0e114fa43248c4528c4d85cf6459 2025-12-04T09:54:55.3185150Z deleted: sha256:144567b58c97f1b5a75d54c3a1514a26ad5ed2dc7988e166316e324a76e76aa6 2025-12-04T09:54:55.3185886Z deleted: sha256:049fbb3123a92952eba68198938b7b29203fcb3d74dfc74e67f9929999576f19 2025-12-04T09:54:55.3186726Z deleted: sha256:22607d21e148d19a45f3f7afe927a15041b9a5663430f5580cc993c4bf9dd7dc 2025-12-04T09:54:55.3187676Z deleted: sha256:63a29311926064245f6d4a4661513586c8179b1cb460ceb91184c0d1114a742d 2025-12-04T09:54:55.3188436Z deleted: sha256:0072101bc0e6bbe2457f4a2d0de953ae643fc2aa6adc69bb59da9cbf6006c8a9 2025-12-04T09:54:55.3189214Z deleted: sha256:35c8f84104fcf39bc1be4682f46dc8b04cb1c5e4c26f66aac039cb42a3f08128 2025-12-04T09:54:55.3189987Z deleted: sha256:c6da65c085b8e72b2e240a3a7d79c4aa166c247737f5916587e51d40acff1ab8 2025-12-04T09:54:55.3190750Z deleted: sha256:b8b5ced5e8550748e26855aad0bb30864c4c435c64694097e63a9a686ef90ba0 2025-12-04T09:54:55.3191514Z deleted: sha256:9905f824759cfae96d29353305bc8a6130a7316db29dfc3efeb8ba9c7cbb5172 2025-12-04T09:54:55.3192281Z deleted: sha256:2b93a73a0fc57cdcfab2a04a30c9b313d84e5e447d076575212f198508242a00 2025-12-04T09:54:55.3193044Z deleted: sha256:baa73842b47530e5d1421010d895ed9059a567516a7e85b10943a6b5fe6fbe41 2025-12-04T09:54:55.3193781Z deleted: sha256:db79627a360d2322d58fb57c98268676011788961d4682ba034f17c9f5fdd855 2025-12-04T09:54:55.3194543Z deleted: sha256:8439f69d5eab9d42b15df43b938f56c690b18cd5e3884f4b80ddc9edf9e43822 2025-12-04T09:54:55.3195322Z deleted: sha256:e95ccdb10d43bbb6f2d6416bca8e5ad7d76a39985431aa5e40ff192ebeb52883 2025-12-04T09:54:55.3196078Z deleted: sha256:d878081c591fd25e8e494f4565e67f341cb3c459b21596ad8e8a707ff9ca63e6 2025-12-04T09:54:55.3196837Z deleted: sha256:2d594947cf73e58a20e36d52acf22f3d31b7a43291d7017c1ac71ea38c4aa9d1 2025-12-04T09:54:55.3197604Z deleted: sha256:933d315b5f20ac941653d01f2b896cc7710564c9b9c4bf9fd8e850115180c6f9 2025-12-04T09:54:55.3198488Z deleted: sha256:0edebbabed274b9e777a5933aaa21ed58569a412c97423c365eff4645e773981 2025-12-04T09:54:55.3199229Z deleted: sha256:744e1cebe181659b0a93ac857d4c16af4c2ce29f3227368eacd7ef18b69e06a7 2025-12-04T09:54:55.3199967Z deleted: sha256:71393f74ab60b183866b9f5995f85aaa65613968c611c8dc15d043befbac662b 2025-12-04T09:54:55.3200705Z deleted: sha256:741a07a29d03bdecf1fb4dfc8f1654a136064f14e5ded89a1497f476ff24623d 2025-12-04T09:54:55.3202059Z deleted: sha256:83df0b5b82e0926d6046012c3b5098dae9aa37177323a33d6eda09e392844b86 2025-12-04T09:54:55.3202822Z deleted: sha256:65b3d6509a37164c67b36965f4ca77ccb1a8e62bdd8f058e2d0886c82ba0940f 2025-12-04T09:54:55.3203598Z deleted: sha256:a38d6cb5dacc51c3e614f4ed42c05cdc53fc489f6e3dafc3b7d0eb39d3250908 2025-12-04T09:54:55.3204375Z deleted: sha256:c3de4f46ac5273a8218c6c5325247b0bd17f12037cfb49f5101a2b5085746704 2025-12-04T09:54:55.3205129Z deleted: sha256:7f3448d255710b28f79767ded93dcf11e0c1a941c7579d8752f4f0bc589aa0b5 2025-12-04T09:54:55.3205895Z deleted: sha256:fd33fe89098b4a8897870a9a316cd91868ab19f42f800d5b512e679356791e62 2025-12-04T09:54:55.3206664Z deleted: sha256:df2c12f62f5ceb2670bf7cb8aa53f268d11dbbfbf25f4b4d567673702cc3fc30 2025-12-04T09:54:55.3207423Z deleted: sha256:a3d7900a6180e4612708429f9c0edba3e999bb43fd67fd3406ae088ea7016a47 2025-12-04T09:54:55.3208194Z deleted: sha256:f592639a1c1bf8da30a1c21866ee22e76b33ae029ee236c7fd20b1ecc4716775 2025-12-04T09:54:55.3208961Z deleted: sha256:e11e4a9756b2bd4a6e62cde476dc36c6528d71a74e8e7570b271784bc494ab40 2025-12-04T09:54:55.3209730Z deleted: sha256:4c03ab513fc4cf698dd61fc37312e57abf7c4e44d4d4ddab126f38804fddec75 2025-12-04T09:54:55.3210498Z deleted: sha256:b4d3d8b5232e0bac87fa78b9c77997cbe8798faffce2aac259b820ab387ce4ad 2025-12-04T09:54:55.3211281Z deleted: sha256:dfda7f6e86c7217a3eb1fe089352248b5d84ca4f62734abce70beb4a6fc90779 2025-12-04T09:54:55.3212151Z deleted: sha256:00f41b846c66bc26a6381fe3663c611a2f539ae3b7e46e007436afd3335a7517 2025-12-04T09:54:55.3216189Z deleted: sha256:0f35c2638249a4525b29668586b079b4ee01f73bbe33732c9da7cfa7b6afb480 2025-12-04T09:54:55.3216934Z deleted: sha256:7441da072ac3269ddb67441b265fdf82904871335b5c19bb381d3622a95b7745 2025-12-04T09:54:55.3217698Z deleted: sha256:ba70851a3aedb994e8d7f8657f8f108bd90a19272843f389e6a8af48a61db720 2025-12-04T09:54:55.3218475Z deleted: sha256:1984ce6f5cabbd8c86386a8d9b5365923b92bc7ccb9aace9b4fe9570213bbbe1 2025-12-04T09:54:55.3219231Z deleted: sha256:73974f74b436f39a2fdb6461b1e3f7c3e41c73325776fa71d16b942a5b4a365b 2025-12-04T09:54:55.3219702Z 2025-12-04T09:54:55.3219849Z Total reclaimed space: 53.28GB 2025-12-04T09:54:55.3302951Z Post job cleanup. 2025-12-04T09:54:55.3346537Z Post job cleanup. 2025-12-04T09:54:55.5032186Z (node:78837) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead. 2025-12-04T09:54:55.5033171Z (Use `node --trace-deprecation ...` to show where the warning was created) 2025-12-04T09:54:55.5209275Z Post job cleanup. 2025-12-04T09:54:55.5263867Z Post job cleanup. 2025-12-04T09:54:55.6485007Z [command]/usr/bin/git version 2025-12-04T09:54:55.6537947Z git version 2.50.1 2025-12-04T09:54:55.6580902Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/2c31f944-22c7-4215-88c4-58b42768ac04/.gitconfig' 2025-12-04T09:54:55.6596732Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/2c31f944-22c7-4215-88c4-58b42768ac04' before making global git config changes 2025-12-04T09:54:55.6597871Z Adding repository directory to the temporary git global config as a safe directory 2025-12-04T09:54:55.6604102Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2025-12-04T09:54:55.6647836Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2025-12-04T09:54:55.6699492Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2025-12-04T09:54:55.7074464Z Entering 'android/libs/fbjni' 2025-12-04T09:54:55.7144350Z Entering 'third_party/FP16' 2025-12-04T09:54:55.7207940Z Entering 'third_party/FXdiv' 2025-12-04T09:54:55.7282977Z Entering 'third_party/NNPACK' 2025-12-04T09:54:55.7347791Z Entering 'third_party/NVTX' 2025-12-04T09:54:55.7412330Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:54:55.7479701Z Entering 'third_party/XNNPACK' 2025-12-04T09:54:55.7567348Z Entering 'third_party/aiter' 2025-12-04T09:54:55.7632510Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:54:55.7717903Z Entering 'third_party/benchmark' 2025-12-04T09:54:55.7780939Z Entering 'third_party/composable_kernel' 2025-12-04T09:54:55.7861929Z Entering 'third_party/cpp-httplib' 2025-12-04T09:54:55.7934449Z Entering 'third_party/cpuinfo' 2025-12-04T09:54:55.8005240Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:54:55.8071698Z Entering 'third_party/cutlass' 2025-12-04T09:54:55.8162840Z Entering 'third_party/fbgemm' 2025-12-04T09:54:55.8229205Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:54:55.8292814Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:54:55.8371045Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:54:55.8438278Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:54:55.8512665Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:54:55.8577440Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:54:55.8643697Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:54:55.8714675Z Entering 'third_party/flash-attention' 2025-12-04T09:54:55.8783733Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:54:55.8857178Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:54:55.8936155Z Entering 'third_party/flatbuffers' 2025-12-04T09:54:55.9004674Z Entering 'third_party/fmt' 2025-12-04T09:54:55.9078049Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:54:55.9147711Z Entering 'third_party/gloo' 2025-12-04T09:54:55.9219082Z Entering 'third_party/googletest' 2025-12-04T09:54:55.9284154Z Entering 'third_party/ideep' 2025-12-04T09:54:55.9348839Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:54:55.9425493Z Entering 'third_party/ittapi' 2025-12-04T09:54:55.9490771Z Entering 'third_party/kineto' 2025-12-04T09:54:55.9552653Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:54:55.9622156Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:54:55.9688997Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:54:55.9755240Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:54:55.9821882Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:54:55.9887784Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:54:55.9951874Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:54:56.0019558Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:54:56.0092854Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:54:56.0157315Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:54:56.0227327Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:54:56.0288374Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:54:56.0357074Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:54:56.0430791Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:54:56.0493206Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:54:56.0564998Z Entering 'third_party/kleidiai' 2025-12-04T09:54:56.0628768Z Entering 'third_party/mimalloc' 2025-12-04T09:54:56.0692142Z Entering 'third_party/nlohmann' 2025-12-04T09:54:56.0764031Z Entering 'third_party/onnx' 2025-12-04T09:54:56.0857818Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:54:56.0930893Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:54:56.1000616Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:54:56.1068375Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:54:56.1132795Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:54:56.1203290Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:54:56.1269544Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:54:56.1338250Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:54:56.1406588Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:54:56.1469539Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:54:56.1540202Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:54:56.1615667Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:54:56.1702419Z Entering 'third_party/pocketfft' 2025-12-04T09:54:56.1768011Z Entering 'third_party/protobuf' 2025-12-04T09:54:56.1847114Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:54:56.1912837Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:54:56.1978600Z Entering 'third_party/psimd' 2025-12-04T09:54:56.2048982Z Entering 'third_party/pthreadpool' 2025-12-04T09:54:56.2115378Z Entering 'third_party/pybind11' 2025-12-04T09:54:56.2183960Z Entering 'third_party/python-peachpy' 2025-12-04T09:54:56.2251143Z Entering 'third_party/sleef' 2025-12-04T09:54:56.2323793Z Entering 'third_party/tensorpipe' 2025-12-04T09:54:56.2392905Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:54:56.2457497Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:54:56.2524241Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:54:56.2593070Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:54:56.2663915Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:54:56.2755969Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2025-12-04T09:54:56.2785265Z http.https://github.com/.extraheader 2025-12-04T09:54:56.2795221Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2025-12-04T09:54:56.2828988Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2025-12-04T09:54:56.3197325Z Entering 'android/libs/fbjni' 2025-12-04T09:54:56.3248601Z http.https://github.com/.extraheader 2025-12-04T09:54:56.3292589Z Entering 'third_party/FP16' 2025-12-04T09:54:56.3343098Z http.https://github.com/.extraheader 2025-12-04T09:54:56.3382963Z Entering 'third_party/FXdiv' 2025-12-04T09:54:56.3432173Z http.https://github.com/.extraheader 2025-12-04T09:54:56.3471262Z Entering 'third_party/NNPACK' 2025-12-04T09:54:56.3515122Z http.https://github.com/.extraheader 2025-12-04T09:54:56.3558905Z Entering 'third_party/NVTX' 2025-12-04T09:54:56.3607699Z http.https://github.com/.extraheader 2025-12-04T09:54:56.3647020Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:54:56.3691562Z http.https://github.com/.extraheader 2025-12-04T09:54:56.3735571Z Entering 'third_party/XNNPACK' 2025-12-04T09:54:56.3780351Z http.https://github.com/.extraheader 2025-12-04T09:54:56.3844898Z Entering 'third_party/aiter' 2025-12-04T09:54:56.3888721Z http.https://github.com/.extraheader 2025-12-04T09:54:56.3928755Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:54:56.3970181Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4021351Z Entering 'third_party/benchmark' 2025-12-04T09:54:56.4068506Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4110026Z Entering 'third_party/composable_kernel' 2025-12-04T09:54:56.4153261Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4203142Z Entering 'third_party/cpp-httplib' 2025-12-04T09:54:56.4248597Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4293414Z Entering 'third_party/cpuinfo' 2025-12-04T09:54:56.4342952Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4379979Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:54:56.4423264Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4468190Z Entering 'third_party/cutlass' 2025-12-04T09:54:56.4506741Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4560168Z Entering 'third_party/fbgemm' 2025-12-04T09:54:56.4605811Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4653568Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:54:56.4691479Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4736457Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:54:56.4785800Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4834741Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:54:56.4883438Z http.https://github.com/.extraheader 2025-12-04T09:54:56.4928912Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:54:56.4969724Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5020984Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:54:56.5068243Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5105896Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:54:56.5149371Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5187898Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:54:56.5229686Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5270923Z Entering 'third_party/flash-attention' 2025-12-04T09:54:56.5318597Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5364774Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:54:56.5407612Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5455775Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:54:56.5498330Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5550884Z Entering 'third_party/flatbuffers' 2025-12-04T09:54:56.5593497Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5636359Z Entering 'third_party/fmt' 2025-12-04T09:54:56.5682841Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5720204Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:54:56.5765293Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5803530Z Entering 'third_party/gloo' 2025-12-04T09:54:56.5845883Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5887958Z Entering 'third_party/googletest' 2025-12-04T09:54:56.5932297Z http.https://github.com/.extraheader 2025-12-04T09:54:56.5969416Z Entering 'third_party/ideep' 2025-12-04T09:54:56.6012002Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6049439Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:54:56.6094498Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6144095Z Entering 'third_party/ittapi' 2025-12-04T09:54:56.6187135Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6224960Z Entering 'third_party/kineto' 2025-12-04T09:54:56.6268620Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6306491Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:54:56.6349269Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6388749Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:54:56.6432652Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6477699Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:54:56.6522651Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6569554Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:54:56.6622820Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6667209Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:54:56.6708548Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6751704Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:54:56.6796443Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6841506Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:54:56.6886061Z http.https://github.com/.extraheader 2025-12-04T09:54:56.6928605Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:54:56.6973915Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7013493Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:54:56.7058182Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7097543Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:54:56.7142418Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7189470Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:54:56.7237949Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7282100Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:54:56.7326427Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7369510Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:54:56.7410602Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7456090Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:54:56.7505568Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7549292Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:54:56.7596901Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7639332Z Entering 'third_party/kleidiai' 2025-12-04T09:54:56.7684296Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7730836Z Entering 'third_party/mimalloc' 2025-12-04T09:54:56.7779289Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7819731Z Entering 'third_party/nlohmann' 2025-12-04T09:54:56.7868428Z http.https://github.com/.extraheader 2025-12-04T09:54:56.7909447Z Entering 'third_party/onnx' 2025-12-04T09:54:56.7952370Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8017849Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:54:56.8063859Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8110733Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:54:56.8164781Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8207868Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:54:56.8255994Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8299548Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:54:56.8341905Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8381080Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:54:56.8424282Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8469694Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:54:56.8511098Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8558685Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:54:56.8601791Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8644253Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:54:56.8685581Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8722881Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:54:56.8767376Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8809200Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:54:56.8855576Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8896161Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:54:56.8938618Z http.https://github.com/.extraheader 2025-12-04T09:54:56.8983500Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:54:56.9031210Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9096887Z Entering 'third_party/pocketfft' 2025-12-04T09:54:56.9143526Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9181678Z Entering 'third_party/protobuf' 2025-12-04T09:54:56.9231172Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9272515Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:54:56.9325619Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9365089Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:54:56.9406917Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9450011Z Entering 'third_party/psimd' 2025-12-04T09:54:56.9499798Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9548588Z Entering 'third_party/pthreadpool' 2025-12-04T09:54:56.9591166Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9629544Z Entering 'third_party/pybind11' 2025-12-04T09:54:56.9672047Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9716738Z Entering 'third_party/python-peachpy' 2025-12-04T09:54:56.9759116Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9808469Z Entering 'third_party/sleef' 2025-12-04T09:54:56.9861227Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9903437Z Entering 'third_party/tensorpipe' 2025-12-04T09:54:56.9945849Z http.https://github.com/.extraheader 2025-12-04T09:54:56.9982615Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:54:57.0028842Z http.https://github.com/.extraheader 2025-12-04T09:54:57.0068497Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:54:57.0116707Z http.https://github.com/.extraheader 2025-12-04T09:54:57.0155748Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:54:57.0197722Z http.https://github.com/.extraheader 2025-12-04T09:54:57.0236589Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:54:57.0291813Z http.https://github.com/.extraheader 2025-12-04T09:54:57.0331129Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:54:57.0373984Z http.https://github.com/.extraheader 2025-12-04T09:54:57.0436118Z [command]/usr/bin/git config --local --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.0475557Z [command]/usr/bin/git submodule foreach --recursive git config --local --show-origin --name-only --get-regexp remote.origin.url 2025-12-04T09:54:57.0857327Z Entering 'android/libs/fbjni' 2025-12-04T09:54:57.0887614Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2025-12-04T09:54:57.0906164Z Entering 'third_party/FP16' 2025-12-04T09:54:57.0936857Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2025-12-04T09:54:57.0953205Z Entering 'third_party/FXdiv' 2025-12-04T09:54:57.0984093Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2025-12-04T09:54:57.1001909Z Entering 'third_party/NNPACK' 2025-12-04T09:54:57.1034112Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2025-12-04T09:54:57.1056398Z Entering 'third_party/NVTX' 2025-12-04T09:54:57.1092809Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config remote.origin.url 2025-12-04T09:54:57.1104367Z Entering 'third_party/VulkanMemoryAllocator' 2025-12-04T09:54:57.1134653Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2025-12-04T09:54:57.1162223Z Entering 'third_party/XNNPACK' 2025-12-04T09:54:57.1189635Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2025-12-04T09:54:57.1228286Z Entering 'third_party/aiter' 2025-12-04T09:54:57.1257716Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/config remote.origin.url 2025-12-04T09:54:57.1282695Z Entering 'third_party/aiter/3rdparty/composable_kernel' 2025-12-04T09:54:57.1311317Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/modules/3rdparty/composable_kernel/config remote.origin.url 2025-12-04T09:54:57.1342459Z Entering 'third_party/benchmark' 2025-12-04T09:54:57.1372199Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:54:57.1388872Z Entering 'third_party/composable_kernel' 2025-12-04T09:54:57.1419683Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config remote.origin.url 2025-12-04T09:54:57.1447235Z Entering 'third_party/cpp-httplib' 2025-12-04T09:54:57.1477985Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2025-12-04T09:54:57.1497171Z Entering 'third_party/cpuinfo' 2025-12-04T09:54:57.1530986Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2025-12-04T09:54:57.1549653Z Entering 'third_party/cudnn_frontend' 2025-12-04T09:54:57.1579084Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2025-12-04T09:54:57.1596824Z Entering 'third_party/cutlass' 2025-12-04T09:54:57.1627202Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2025-12-04T09:54:57.1658706Z Entering 'third_party/fbgemm' 2025-12-04T09:54:57.1688728Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2025-12-04T09:54:57.1707489Z Entering 'third_party/fbgemm/external/asmjit' 2025-12-04T09:54:57.1740023Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/asmjit/config remote.origin.url 2025-12-04T09:54:57.1762019Z Entering 'third_party/fbgemm/external/composable_kernel' 2025-12-04T09:54:57.1793816Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/composable_kernel/config remote.origin.url 2025-12-04T09:54:57.1825604Z Entering 'third_party/fbgemm/external/cpuinfo' 2025-12-04T09:54:57.1855331Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cpuinfo/config remote.origin.url 2025-12-04T09:54:57.1872944Z Entering 'third_party/fbgemm/external/cutlass' 2025-12-04T09:54:57.1901450Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cutlass/config remote.origin.url 2025-12-04T09:54:57.1931632Z Entering 'third_party/fbgemm/external/googletest' 2025-12-04T09:54:57.1958109Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/googletest/config remote.origin.url 2025-12-04T09:54:57.1982753Z Entering 'third_party/fbgemm/external/hipify_torch' 2025-12-04T09:54:57.2010520Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/hipify_torch/config remote.origin.url 2025-12-04T09:54:57.2033610Z Entering 'third_party/fbgemm/external/json' 2025-12-04T09:54:57.2061529Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/json/config remote.origin.url 2025-12-04T09:54:57.2090958Z Entering 'third_party/flash-attention' 2025-12-04T09:54:57.2114869Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config remote.origin.url 2025-12-04T09:54:57.2135140Z Entering 'third_party/flash-attention/csrc/composable_kernel' 2025-12-04T09:54:57.2164192Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config remote.origin.url 2025-12-04T09:54:57.2188399Z Entering 'third_party/flash-attention/csrc/cutlass' 2025-12-04T09:54:57.2218776Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config remote.origin.url 2025-12-04T09:54:57.2249393Z Entering 'third_party/flatbuffers' 2025-12-04T09:54:57.2280793Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2025-12-04T09:54:57.2303546Z Entering 'third_party/fmt' 2025-12-04T09:54:57.2333790Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2025-12-04T09:54:57.2352765Z Entering 'third_party/gemmlowp/gemmlowp' 2025-12-04T09:54:57.2382417Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2025-12-04T09:54:57.2402297Z Entering 'third_party/gloo' 2025-12-04T09:54:57.2429999Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2025-12-04T09:54:57.2450151Z Entering 'third_party/googletest' 2025-12-04T09:54:57.2484423Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:54:57.2503413Z Entering 'third_party/ideep' 2025-12-04T09:54:57.2537477Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2025-12-04T09:54:57.2554472Z Entering 'third_party/ideep/mkl-dnn' 2025-12-04T09:54:57.2589499Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2025-12-04T09:54:57.2614816Z Entering 'third_party/ittapi' 2025-12-04T09:54:57.2643342Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2025-12-04T09:54:57.2661620Z Entering 'third_party/kineto' 2025-12-04T09:54:57.2692385Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2025-12-04T09:54:57.2717597Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2025-12-04T09:54:57.2744810Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2025-12-04T09:54:57.2765195Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2025-12-04T09:54:57.2793853Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2025-12-04T09:54:57.2815296Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2025-12-04T09:54:57.2843973Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2025-12-04T09:54:57.2864186Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2025-12-04T09:54:57.2893211Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2025-12-04T09:54:57.2912177Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2025-12-04T09:54:57.2941062Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2025-12-04T09:54:57.2955686Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2025-12-04T09:54:57.2989902Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2025-12-04T09:54:57.3008999Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2025-12-04T09:54:57.3044793Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2025-12-04T09:54:57.3063949Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2025-12-04T09:54:57.3092558Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:54:57.3110962Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2025-12-04T09:54:57.3141700Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2025-12-04T09:54:57.3160773Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2025-12-04T09:54:57.3189211Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2025-12-04T09:54:57.3205626Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp' 2025-12-04T09:54:57.3244474Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/config remote.origin.url 2025-12-04T09:54:57.3263371Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:54:57.3296981Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2025-12-04T09:54:57.3317414Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:54:57.3346390Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2025-12-04T09:54:57.3373014Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2025-12-04T09:54:57.3400273Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2025-12-04T09:54:57.3421445Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2025-12-04T09:54:57.3448986Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2025-12-04T09:54:57.3470340Z Entering 'third_party/kleidiai' 2025-12-04T09:54:57.3503037Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config remote.origin.url 2025-12-04T09:54:57.3524445Z Entering 'third_party/mimalloc' 2025-12-04T09:54:57.3555647Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2025-12-04T09:54:57.3576496Z Entering 'third_party/nlohmann' 2025-12-04T09:54:57.3604631Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2025-12-04T09:54:57.3626861Z Entering 'third_party/onnx' 2025-12-04T09:54:57.3661526Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2025-12-04T09:54:57.3701720Z Entering 'third_party/onnx/third_party/pybind11' 2025-12-04T09:54:57.3739009Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:54:57.3764804Z Entering 'third_party/opentelemetry-cpp' 2025-12-04T09:54:57.3801566Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2025-12-04T09:54:57.3823450Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2025-12-04T09:54:57.3852381Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:54:57.3869610Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2025-12-04T09:54:57.3901510Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:54:57.3922612Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2025-12-04T09:54:57.3949473Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2025-12-04T09:54:57.3967773Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2025-12-04T09:54:57.3995603Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2025-12-04T09:54:57.4018448Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2025-12-04T09:54:57.4054251Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2025-12-04T09:54:57.4071705Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2025-12-04T09:54:57.4100383Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2025-12-04T09:54:57.4118134Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2025-12-04T09:54:57.4147815Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2025-12-04T09:54:57.4168116Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2025-12-04T09:54:57.4194796Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2025-12-04T09:54:57.4215257Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2025-12-04T09:54:57.4247159Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2025-12-04T09:54:57.4268298Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2025-12-04T09:54:57.4296235Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2025-12-04T09:54:57.4344929Z Entering 'third_party/pocketfft' 2025-12-04T09:54:57.4376753Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2025-12-04T09:54:57.4393643Z Entering 'third_party/protobuf' 2025-12-04T09:54:57.4429027Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2025-12-04T09:54:57.4451930Z Entering 'third_party/protobuf/third_party/benchmark' 2025-12-04T09:54:57.4486474Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2025-12-04T09:54:57.4503357Z Entering 'third_party/protobuf/third_party/googletest' 2025-12-04T09:54:57.4532602Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:54:57.4552479Z Entering 'third_party/psimd' 2025-12-04T09:54:57.4583508Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2025-12-04T09:54:57.4601269Z Entering 'third_party/pthreadpool' 2025-12-04T09:54:57.4630156Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2025-12-04T09:54:57.4655882Z Entering 'third_party/pybind11' 2025-12-04T09:54:57.4683555Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:54:57.4701408Z Entering 'third_party/python-peachpy' 2025-12-04T09:54:57.4732724Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2025-12-04T09:54:57.4750533Z Entering 'third_party/sleef' 2025-12-04T09:54:57.4784590Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2025-12-04T09:54:57.4804248Z Entering 'third_party/tensorpipe' 2025-12-04T09:54:57.4834033Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2025-12-04T09:54:57.4858611Z Entering 'third_party/tensorpipe/third_party/googletest' 2025-12-04T09:54:57.4887218Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2025-12-04T09:54:57.4910105Z Entering 'third_party/tensorpipe/third_party/libnop' 2025-12-04T09:54:57.4940193Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2025-12-04T09:54:57.4957641Z Entering 'third_party/tensorpipe/third_party/libuv' 2025-12-04T09:54:57.4988352Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2025-12-04T09:54:57.5005739Z Entering 'third_party/tensorpipe/third_party/pybind11' 2025-12-04T09:54:57.5036155Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2025-12-04T09:54:57.5055006Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2025-12-04T09:54:57.5089121Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2025-12-04T09:54:57.5132784Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5164285Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5190902Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5222296Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5250173Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5282215Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5309914Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5339233Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5366719Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/modules/3rdparty/composable_kernel/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5393587Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5426169Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5453229Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5484603Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5512567Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5542738Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5570471Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5597654Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/asmjit/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5624011Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/composable_kernel/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5655795Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cpuinfo/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5682290Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cutlass/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5714285Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5743409Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/hipify_torch/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5771794Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/json/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5797138Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5824807Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5852570Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5877293Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5908799Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5936278Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5965490Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.5990601Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6018111Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6045372Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6072074Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6099294Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6126641Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6157746Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6184631Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6215249Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6242204Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6269777Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6304765Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6331811Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6363814Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6390227Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6417511Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6445849Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/civetweb/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6475342Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/prometheus-cpp/modules/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6500704Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6528961Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6554268Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6579572Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6611308Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6637612Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6668104Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6694054Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6721824Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6748762Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6777723Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6801594Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6829224Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6855981Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6884391Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6920552Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6956728Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.6988561Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7015831Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7043953Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7071817Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7097976Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7125144Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7155165Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7180333Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7212689Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7235959Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7262568Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7290153Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7317134Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7344825Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7370883Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7403627Z [command]/usr/bin/git config --file /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config --name-only --get-regexp ^includeIf\.gitdir: 2025-12-04T09:54:57.7514734Z A job completed hook has been configured by the self-hosted runner administrator 2025-12-04T09:54:57.7537354Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2025-12-04T09:54:57.7543177Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2025-12-04T09:54:57.7543595Z ##[endgroup] 2025-12-04T09:54:57.7641525Z [!ALERT!] Swap in detected! [!ALERT!] 2025-12-04T09:55:10.4926639Z [!ALERT!] Swap out detected [!ALERT!] 2025-12-04T09:55:31.8082532Z Cleaning up orphan processes