2024-08-20T21:41:37.7643275Z Current runner version: '2.319.1' 2024-08-20T21:41:37.7650126Z Runner name: 'i-0f00d0841bbe75141' 2024-08-20T21:41:37.7650966Z Runner group name: 'Default' 2024-08-20T21:41:37.7651669Z Machine name: 'ip-10-0-13-192' 2024-08-20T21:41:37.7668152Z Testing runner upgrade compatibility 2024-08-20T21:41:37.9044196Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T21:41:37.9046412Z Actions: read 2024-08-20T21:41:37.9047015Z Attestations: read 2024-08-20T21:41:37.9047491Z Checks: read 2024-08-20T21:41:37.9047937Z Contents: read 2024-08-20T21:41:37.9048401Z Deployments: read 2024-08-20T21:41:37.9048873Z Discussions: read 2024-08-20T21:41:37.9049326Z Issues: read 2024-08-20T21:41:37.9049758Z Metadata: read 2024-08-20T21:41:37.9050209Z Packages: read 2024-08-20T21:41:37.9050655Z Pages: read 2024-08-20T21:41:37.9051134Z PullRequests: read 2024-08-20T21:41:37.9051591Z RepositoryProjects: read 2024-08-20T21:41:37.9052135Z SecurityEvents: read 2024-08-20T21:41:37.9052654Z Statuses: read 2024-08-20T21:41:37.9053285Z ##[endgroup] 2024-08-20T21:41:37.9056830Z Secret source: Actions 2024-08-20T21:41:37.9057426Z Prepare workflow directory 2024-08-20T21:41:37.9948242Z Prepare all required actions 2024-08-20T21:41:38.0101390Z Getting action download info 2024-08-20T21:41:38.1721841Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T21:41:38.5316286Z Download action repository 'pytorch/pytorch@main' (SHA:15b5a0b67fc3f34fb0bf1afa6f91e0c4c2b7fd8d) 2024-08-20T21:41:41.4131119Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T21:41:41.5497876Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T21:41:41.8663280Z Getting action download info 2024-08-20T21:41:41.9472364Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T21:41:42.1241373Z Getting action download info 2024-08-20T21:41:42.2324929Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T21:41:42.3679023Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-20T21:41:42.3681311Z ##[group] Inputs 2024-08-20T21:41:42.3681724Z build-environment: linux-jammy-py3.8-gcc11-build 2024-08-20T21:41:42.3702155Z test-matrix: {"include": [{"config": "inductor_avx512", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_avx512", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_amp_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_torchbench_cpu_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.24xl.spr-metal"}, {"config": "inductor_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "inductor_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_huggingface_freezing_avx2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}]} 2024-08-20T21:41:42.3723746Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:41:42.3724874Z sync-tag: 2024-08-20T21:41:42.3725792Z timeout-minutes: 240 2024-08-20T21:41:42.3726083Z use-gha: 2024-08-20T21:41:42.3726312Z dashboard-tag: 2024-08-20T21:41:42.3726588Z s3-bucket: gha-artifacts 2024-08-20T21:41:42.3726893Z aws-role-to-assume: 2024-08-20T21:41:42.3727180Z ##[endgroup] 2024-08-20T21:41:42.3728189Z Complete job name: linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_amp_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:41:42.4492378Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T21:41:42.4625399Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T21:41:42.4634581Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:41:42.4635270Z ##[endgroup] 2024-08-20T21:41:43.5215522Z Runner Type: amz2023.linux.12xlarge 2024-08-20T21:41:43.5216045Z Instance Type: c5.12xlarge 2024-08-20T21:41:43.5216758Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T21:41:43.5217313Z AMI ID: ami-06c68f701d8090592 2024-08-20T21:41:48.8465364Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T21:41:48.8465948Z with: 2024-08-20T21:41:48.8466692Z github-secret: *** 2024-08-20T21:41:48.8467689Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T21:41:48.8468753Z activate-with-label: false 2024-08-20T21:41:48.8469101Z label: with-ssh 2024-08-20T21:41:48.8469406Z remove-existing-keys: true 2024-08-20T21:41:48.8469746Z fail-silently: true 2024-08-20T21:41:48.8470040Z env: 2024-08-20T21:41:48.8470401Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:41:48.8470710Z ##[endgroup] 2024-08-20T21:41:48.9269115Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T21:41:48.9271573Z ciflow reference detected, attempting to extract PR number 2024-08-20T21:41:49.1841131Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-20T21:41:49.2499549Z No SSH keys found for user pytorch-bot[bot] 2024-08-20T21:41:49.2500208Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T21:41:49.3172880Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T21:41:49.3187103Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T21:41:49.3211430Z Login using: ssh ec2-user@ec2-54-166-255-47.compute-1.amazonaws.com 2024-08-20T21:41:49.3212254Z All testing is done inside the container, to start an interactive session run: 2024-08-20T21:41:49.3213111Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T21:41:49.3330266Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T21:41:49.3330827Z with: 2024-08-20T21:41:49.3331088Z submodules: recursive 2024-08-20T21:41:49.3331383Z fetch-depth: 0 2024-08-20T21:41:49.3331867Z env: 2024-08-20T21:41:49.3332201Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:41:49.3332616Z ##[endgroup] 2024-08-20T21:41:49.3527537Z ##[group]Run retry () { 2024-08-20T21:41:49.3527974Z retry () { 2024-08-20T21:41:49.3528482Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T21:41:49.3529148Z } 2024-08-20T21:41:49.3529509Z echo "${GITHUB_WORKSPACE}" 2024-08-20T21:41:49.3530045Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T21:41:49.3530533Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T21:41:49.3531037Z else 2024-08-20T21:41:49.3531670Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T21:41:49.3532172Z fi 2024-08-20T21:41:49.3532618Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T21:41:49.3541612Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:41:49.3542305Z env: 2024-08-20T21:41:49.3542639Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:41:49.3543061Z NO_SUDO: 2024-08-20T21:41:49.3543720Z ##[endgroup] 2024-08-20T21:41:49.3612132Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:41:49.4896703Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T21:41:49.4897111Z with: 2024-08-20T21:41:49.4897409Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:41:49.4897932Z fetch-depth: 0 2024-08-20T21:41:49.4898389Z submodules: recursive 2024-08-20T21:41:49.4898726Z quiet-checkout: true 2024-08-20T21:41:49.4899070Z repository: pytorch/pytorch 2024-08-20T21:41:49.4899563Z token: *** 2024-08-20T21:41:49.4899835Z ssh-strict: true 2024-08-20T21:41:49.4900154Z persist-credentials: true 2024-08-20T21:41:49.4900506Z clean: true 2024-08-20T21:41:49.4900816Z sparse-checkout-cone-mode: true 2024-08-20T21:41:49.4901411Z lfs: false 2024-08-20T21:41:49.4901703Z set-safe-directory: true 2024-08-20T21:41:49.4902016Z env: 2024-08-20T21:41:49.4902279Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:41:49.4902617Z ##[endgroup] 2024-08-20T21:41:49.5812115Z Syncing repository: pytorch/pytorch 2024-08-20T21:41:49.5813926Z ##[group]Getting Git version info 2024-08-20T21:41:49.5814693Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T21:41:49.5815568Z [command]/usr/bin/git version 2024-08-20T21:41:49.5815934Z git version 2.40.1 2024-08-20T21:41:49.5817318Z ##[endgroup] 2024-08-20T21:41:49.5828792Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/67d30710-bad3-46f9-80eb-e7e484274d66' before making global git config changes 2024-08-20T21:41:49.5830098Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T21:41:49.5831293Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:41:49.5861106Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T21:41:49.5864708Z ##[group]Initializing the repository 2024-08-20T21:41:49.5867266Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:41:49.5898854Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T21:41:49.5899768Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T21:41:49.5900583Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T21:41:49.5901162Z hint: 2024-08-20T21:41:49.5901730Z hint: git config --global init.defaultBranch 2024-08-20T21:41:49.5902203Z hint: 2024-08-20T21:41:49.5902727Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T21:41:49.5903604Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T21:41:49.5904412Z hint: 2024-08-20T21:41:49.5904737Z hint: git branch -m 2024-08-20T21:41:49.5905498Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T21:41:49.5907091Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T21:41:49.5936221Z ##[endgroup] 2024-08-20T21:41:49.5936784Z ##[group]Disabling automatic garbage collection 2024-08-20T21:41:49.5938154Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T21:41:49.5967502Z ##[endgroup] 2024-08-20T21:41:49.5968011Z ##[group]Setting up auth 2024-08-20T21:41:49.5971969Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T21:41:49.6001236Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T21:41:49.6283626Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T21:41:49.6311470Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T21:41:49.6570524Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T21:41:49.6619558Z ##[endgroup] 2024-08-20T21:41:49.6620108Z ##[group]Fetching the repository 2024-08-20T21:41:49.6624734Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T21:41:52.6002738Z remote: Enumerating objects: 1008510 2024-08-20T21:41:52.6004337Z remote: Enumerating objects: 1010152, done. 2024-08-20T21:41:52.6005093Z remote: Counting objects: 0% (1/1642) 2024-08-20T21:41:52.6005621Z remote: Counting objects: 1% (17/1642) 2024-08-20T21:41:52.6006300Z remote: Counting objects: 2% (33/1642) 2024-08-20T21:41:52.6007208Z remote: Counting objects: 3% (50/1642) 2024-08-20T21:41:52.6007708Z remote: Counting objects: 4% (66/1642) 2024-08-20T21:41:52.6008226Z remote: Counting objects: 5% (83/1642) 2024-08-20T21:41:52.6008742Z remote: Counting objects: 6% (99/1642) 2024-08-20T21:41:52.6009254Z remote: Counting objects: 7% (115/1642) 2024-08-20T21:41:52.6009755Z remote: Counting objects: 8% (132/1642) 2024-08-20T21:41:52.6010276Z remote: Counting objects: 9% (148/1642) 2024-08-20T21:41:52.6010788Z remote: Counting objects: 10% (165/1642) 2024-08-20T21:41:52.6011299Z remote: Counting objects: 11% (181/1642) 2024-08-20T21:41:52.6011793Z remote: Counting objects: 12% (198/1642) 2024-08-20T21:41:52.6012311Z remote: Counting objects: 13% (214/1642) 2024-08-20T21:41:52.6012820Z remote: Counting objects: 14% (230/1642) 2024-08-20T21:41:52.6013336Z remote: Counting objects: 15% (247/1642) 2024-08-20T21:41:52.6013830Z remote: Counting objects: 16% (263/1642) 2024-08-20T21:41:52.6014340Z remote: Counting objects: 17% (280/1642) 2024-08-20T21:41:52.6014847Z remote: Counting objects: 18% (296/1642) 2024-08-20T21:41:52.6015353Z remote: Counting objects: 19% (312/1642) 2024-08-20T21:41:52.6015842Z remote: Counting objects: 20% (329/1642) 2024-08-20T21:41:52.6016356Z remote: Counting objects: 21% (345/1642) 2024-08-20T21:41:52.6016867Z remote: Counting objects: 22% (362/1642) 2024-08-20T21:41:52.6017376Z remote: Counting objects: 23% (378/1642) 2024-08-20T21:41:52.6018016Z remote: Counting objects: 24% (395/1642) 2024-08-20T21:41:52.6018635Z remote: Counting objects: 25% (411/1642) 2024-08-20T21:41:52.6019156Z remote: Counting objects: 26% (427/1642) 2024-08-20T21:41:52.6019676Z remote: Counting objects: 27% (444/1642) 2024-08-20T21:41:52.6020349Z remote: Counting objects: 28% (460/1642) 2024-08-20T21:41:52.6020880Z remote: Counting objects: 29% (477/1642) 2024-08-20T21:41:52.6021399Z remote: Counting objects: 30% (493/1642) 2024-08-20T21:41:52.6021914Z remote: Counting objects: 31% (510/1642) 2024-08-20T21:41:52.6022411Z remote: Counting objects: 32% (526/1642) 2024-08-20T21:41:52.6022929Z remote: Counting objects: 33% (542/1642) 2024-08-20T21:41:52.6023442Z remote: Counting objects: 34% (559/1642) 2024-08-20T21:41:52.6023947Z remote: Counting objects: 35% (575/1642) 2024-08-20T21:41:52.6024435Z remote: Counting objects: 36% (592/1642) 2024-08-20T21:41:52.6024948Z remote: Counting objects: 37% (608/1642) 2024-08-20T21:41:52.6025462Z remote: Counting objects: 38% (624/1642) 2024-08-20T21:41:52.6025971Z remote: Counting objects: 39% (641/1642) 2024-08-20T21:41:52.6026500Z remote: Counting objects: 40% (657/1642) 2024-08-20T21:41:52.6027018Z remote: Counting objects: 41% (674/1642) 2024-08-20T21:41:52.6027533Z remote: Counting objects: 42% (690/1642) 2024-08-20T21:41:52.6028043Z remote: Counting objects: 43% (707/1642) 2024-08-20T21:41:52.6028533Z remote: Counting objects: 44% (723/1642) 2024-08-20T21:41:52.6029046Z remote: Counting objects: 45% (739/1642) 2024-08-20T21:41:52.6029555Z remote: Counting objects: 46% (756/1642) 2024-08-20T21:41:52.6030062Z remote: Counting objects: 47% (772/1642) 2024-08-20T21:41:52.6030553Z remote: Counting objects: 48% (789/1642) 2024-08-20T21:41:52.6031062Z remote: Counting objects: 49% (805/1642) 2024-08-20T21:41:52.6031570Z remote: Counting objects: 50% (821/1642) 2024-08-20T21:41:52.6032076Z remote: Counting objects: 51% (838/1642) 2024-08-20T21:41:52.6032563Z remote: Counting objects: 52% (854/1642) 2024-08-20T21:41:52.6033077Z remote: Counting objects: 53% (871/1642) 2024-08-20T21:41:52.6033591Z remote: Counting objects: 54% (887/1642) 2024-08-20T21:41:52.6034190Z remote: Counting objects: 55% (904/1642) 2024-08-20T21:41:52.6034678Z remote: Counting objects: 56% (920/1642) 2024-08-20T21:41:52.6035188Z remote: Counting objects: 57% (936/1642) 2024-08-20T21:41:52.6035703Z remote: Counting objects: 58% (953/1642) 2024-08-20T21:41:52.6036217Z remote: Counting objects: 59% (969/1642) 2024-08-20T21:41:52.6036707Z remote: Counting objects: 60% (986/1642) 2024-08-20T21:41:52.6037230Z remote: Counting objects: 61% (1002/1642) 2024-08-20T21:41:52.6037764Z remote: Counting objects: 62% (1019/1642) 2024-08-20T21:41:52.6038309Z remote: Counting objects: 63% (1035/1642) 2024-08-20T21:41:52.6038842Z remote: Counting objects: 64% (1051/1642) 2024-08-20T21:41:52.6039356Z remote: Counting objects: 65% (1068/1642) 2024-08-20T21:41:52.6039886Z remote: Counting objects: 66% (1084/1642) 2024-08-20T21:41:52.6040417Z remote: Counting objects: 67% (1101/1642) 2024-08-20T21:41:52.6040947Z remote: Counting objects: 68% (1117/1642) 2024-08-20T21:41:52.6041458Z remote: Counting objects: 69% (1133/1642) 2024-08-20T21:41:52.6041987Z remote: Counting objects: 70% (1150/1642) 2024-08-20T21:41:52.6042510Z remote: Counting objects: 71% (1166/1642) 2024-08-20T21:41:52.6043041Z remote: Counting objects: 72% (1183/1642) 2024-08-20T21:41:52.6043778Z remote: Counting objects: 73% (1199/1642) 2024-08-20T21:41:52.6044312Z remote: Counting objects: 74% (1216/1642) 2024-08-20T21:41:52.6044848Z remote: Counting objects: 75% (1232/1642) 2024-08-20T21:41:52.6045380Z remote: Counting objects: 76% (1248/1642) 2024-08-20T21:41:52.6045894Z remote: Counting objects: 77% (1265/1642) 2024-08-20T21:41:52.6046424Z remote: Counting objects: 78% (1281/1642) 2024-08-20T21:41:52.6047079Z remote: Counting objects: 79% (1298/1642) 2024-08-20T21:41:52.6047626Z remote: Counting objects: 80% (1314/1642) 2024-08-20T21:41:52.6048141Z remote: Counting objects: 81% (1331/1642) 2024-08-20T21:41:52.6048672Z remote: Counting objects: 82% (1347/1642) 2024-08-20T21:41:52.6049204Z remote: Counting objects: 83% (1363/1642) 2024-08-20T21:41:52.6049742Z remote: Counting objects: 84% (1380/1642) 2024-08-20T21:41:52.6050262Z remote: Counting objects: 85% (1396/1642) 2024-08-20T21:41:52.6050788Z remote: Counting objects: 86% (1413/1642) 2024-08-20T21:41:52.6051315Z remote: Counting objects: 87% (1429/1642) 2024-08-20T21:41:52.6051841Z remote: Counting objects: 88% (1445/1642) 2024-08-20T21:41:52.6052351Z remote: Counting objects: 89% (1462/1642) 2024-08-20T21:41:52.6052876Z remote: Counting objects: 90% (1478/1642) 2024-08-20T21:41:52.6053406Z remote: Counting objects: 91% (1495/1642) 2024-08-20T21:41:52.6053935Z remote: Counting objects: 92% (1511/1642) 2024-08-20T21:41:52.6054452Z remote: Counting objects: 93% (1528/1642) 2024-08-20T21:41:52.6054979Z remote: Counting objects: 94% (1544/1642) 2024-08-20T21:41:52.6055499Z remote: Counting objects: 95% (1560/1642) 2024-08-20T21:41:52.6056007Z remote: Counting objects: 96% (1577/1642) 2024-08-20T21:41:52.6056532Z remote: Counting objects: 97% (1593/1642) 2024-08-20T21:41:52.6057054Z remote: Counting objects: 98% (1610/1642) 2024-08-20T21:41:52.6057585Z remote: Counting objects: 99% (1626/1642) 2024-08-20T21:41:52.6058188Z remote: Counting objects: 100% (1642/1642) 2024-08-20T21:41:52.6058760Z remote: Counting objects: 100% (1642/1642), done. 2024-08-20T21:41:52.6196519Z remote: Compressing objects: 0% (1/835) 2024-08-20T21:41:52.6461664Z remote: Compressing objects: 1% (9/835) 2024-08-20T21:41:52.6879053Z remote: Compressing objects: 2% (17/835) 2024-08-20T21:41:52.7547330Z remote: Compressing objects: 3% (26/835) 2024-08-20T21:41:52.8323750Z remote: Compressing objects: 4% (34/835) 2024-08-20T21:41:52.8797139Z remote: Compressing objects: 5% (42/835) 2024-08-20T21:41:52.9254598Z remote: Compressing objects: 6% (51/835) 2024-08-20T21:41:52.9619565Z remote: Compressing objects: 7% (59/835) 2024-08-20T21:41:52.9961521Z remote: Compressing objects: 8% (67/835) 2024-08-20T21:41:53.0273378Z remote: Compressing objects: 9% (76/835) 2024-08-20T21:41:53.0500656Z remote: Compressing objects: 10% (84/835) 2024-08-20T21:41:53.0733851Z remote: Compressing objects: 11% (92/835) 2024-08-20T21:41:53.0959360Z remote: Compressing objects: 12% (101/835) 2024-08-20T21:41:53.1105064Z remote: Compressing objects: 13% (109/835) 2024-08-20T21:41:53.1228213Z remote: Compressing objects: 14% (117/835) 2024-08-20T21:41:53.1296798Z remote: Compressing objects: 15% (126/835) 2024-08-20T21:41:53.1386726Z remote: Compressing objects: 16% (134/835) 2024-08-20T21:41:53.1478405Z remote: Compressing objects: 17% (142/835) 2024-08-20T21:41:53.1527973Z remote: Compressing objects: 18% (151/835) 2024-08-20T21:41:53.1569726Z remote: Compressing objects: 19% (159/835) 2024-08-20T21:41:53.1607654Z remote: Compressing objects: 20% (167/835) 2024-08-20T21:41:53.1634904Z remote: Compressing objects: 21% (176/835) 2024-08-20T21:41:53.1644279Z remote: Compressing objects: 22% (184/835) 2024-08-20T21:41:53.1644874Z remote: Compressing objects: 23% (193/835) 2024-08-20T21:41:53.1648713Z remote: Compressing objects: 24% (201/835) 2024-08-20T21:41:53.1660177Z remote: Compressing objects: 25% (209/835) 2024-08-20T21:41:53.1660774Z remote: Compressing objects: 26% (218/835) 2024-08-20T21:41:53.1676476Z remote: Compressing objects: 27% (226/835) 2024-08-20T21:41:53.1684578Z remote: Compressing objects: 28% (234/835) 2024-08-20T21:41:53.1701434Z remote: Compressing objects: 29% (243/835) 2024-08-20T21:41:53.1708457Z remote: Compressing objects: 30% (251/835) 2024-08-20T21:41:53.1721638Z remote: Compressing objects: 31% (259/835) 2024-08-20T21:41:53.1727928Z remote: Compressing objects: 32% (268/835) 2024-08-20T21:41:53.1732595Z remote: Compressing objects: 33% (276/835) 2024-08-20T21:41:53.1743311Z remote: Compressing objects: 34% (284/835) 2024-08-20T21:41:53.1747224Z remote: Compressing objects: 35% (293/835) 2024-08-20T21:41:53.1753775Z remote: Compressing objects: 36% (301/835) 2024-08-20T21:41:53.1758493Z remote: Compressing objects: 37% (309/835) 2024-08-20T21:41:53.1768638Z remote: Compressing objects: 38% (318/835) 2024-08-20T21:41:53.1774568Z remote: Compressing objects: 39% (326/835) 2024-08-20T21:41:53.1781549Z remote: Compressing objects: 40% (334/835) 2024-08-20T21:41:53.1785635Z remote: Compressing objects: 41% (343/835) 2024-08-20T21:41:53.1797294Z remote: Compressing objects: 42% (351/835) 2024-08-20T21:41:53.1801910Z remote: Compressing objects: 43% (360/835) 2024-08-20T21:41:53.1807215Z remote: Compressing objects: 44% (368/835) 2024-08-20T21:41:53.1814143Z remote: Compressing objects: 45% (376/835) 2024-08-20T21:41:53.1819515Z remote: Compressing objects: 46% (385/835) 2024-08-20T21:41:53.1822398Z remote: Compressing objects: 47% (393/835) 2024-08-20T21:41:53.1825779Z remote: Compressing objects: 48% (401/835) 2024-08-20T21:41:53.1829008Z remote: Compressing objects: 49% (410/835) 2024-08-20T21:41:53.1831858Z remote: Compressing objects: 50% (418/835) 2024-08-20T21:41:53.1836832Z remote: Compressing objects: 51% (426/835) 2024-08-20T21:41:53.1839298Z remote: Compressing objects: 52% (435/835) 2024-08-20T21:41:53.1841154Z remote: Compressing objects: 53% (443/835) 2024-08-20T21:41:53.1843791Z remote: Compressing objects: 54% (451/835) 2024-08-20T21:41:53.1846169Z remote: Compressing objects: 55% (460/835) 2024-08-20T21:41:53.1848620Z remote: Compressing objects: 56% (468/835) 2024-08-20T21:41:53.1851513Z remote: Compressing objects: 57% (476/835) 2024-08-20T21:41:53.1852044Z remote: Compressing objects: 58% (485/835) 2024-08-20T21:41:53.1852594Z remote: Compressing objects: 59% (493/835) 2024-08-20T21:41:53.1853141Z remote: Compressing objects: 60% (501/835) 2024-08-20T21:41:53.1853686Z remote: Compressing objects: 61% (510/835) 2024-08-20T21:41:53.1854230Z remote: Compressing objects: 62% (518/835) 2024-08-20T21:41:53.1854756Z remote: Compressing objects: 63% (527/835) 2024-08-20T21:41:53.1855294Z remote: Compressing objects: 64% (535/835) 2024-08-20T21:41:53.1856356Z remote: Compressing objects: 65% (543/835) 2024-08-20T21:41:53.1864354Z remote: Compressing objects: 66% (552/835) 2024-08-20T21:41:53.1869410Z remote: Compressing objects: 67% (560/835) 2024-08-20T21:41:53.1872689Z remote: Compressing objects: 68% (568/835) 2024-08-20T21:41:53.1878535Z remote: Compressing objects: 69% (577/835) 2024-08-20T21:41:53.1881086Z remote: Compressing objects: 70% (585/835) 2024-08-20T21:41:53.1887506Z remote: Compressing objects: 71% (593/835) 2024-08-20T21:41:53.1890179Z remote: Compressing objects: 72% (602/835) 2024-08-20T21:41:53.1893106Z remote: Compressing objects: 73% (610/835) 2024-08-20T21:41:53.1898012Z remote: Compressing objects: 74% (618/835) 2024-08-20T21:41:53.1900586Z remote: Compressing objects: 75% (627/835) 2024-08-20T21:41:53.1903149Z remote: Compressing objects: 76% (635/835) 2024-08-20T21:41:53.1905668Z remote: Compressing objects: 77% (643/835) 2024-08-20T21:41:53.1908543Z remote: Compressing objects: 78% (652/835) 2024-08-20T21:41:53.1911221Z remote: Compressing objects: 79% (660/835) 2024-08-20T21:41:53.1913750Z remote: Compressing objects: 80% (668/835) 2024-08-20T21:41:53.1916257Z remote: Compressing objects: 81% (677/835) 2024-08-20T21:41:53.1918895Z remote: Compressing objects: 82% (685/835) 2024-08-20T21:41:53.1919465Z remote: Compressing objects: 83% (694/835) 2024-08-20T21:41:53.1921411Z remote: Compressing objects: 84% (702/835) 2024-08-20T21:41:53.1924010Z remote: Compressing objects: 85% (710/835) 2024-08-20T21:41:53.1926719Z remote: Compressing objects: 86% (719/835) 2024-08-20T21:41:53.1927259Z remote: Compressing objects: 87% (727/835) 2024-08-20T21:41:53.1929255Z remote: Compressing objects: 88% (735/835) 2024-08-20T21:41:53.1931771Z remote: Compressing objects: 89% (744/835) 2024-08-20T21:41:53.1935365Z remote: Compressing objects: 90% (752/835) 2024-08-20T21:41:53.1936898Z remote: Compressing objects: 91% (760/835) 2024-08-20T21:41:53.1938605Z remote: Compressing objects: 92% (769/835) 2024-08-20T21:41:53.1939980Z remote: Compressing objects: 93% (777/835) 2024-08-20T21:41:53.1940504Z remote: Compressing objects: 94% (785/835) 2024-08-20T21:41:53.1941049Z remote: Compressing objects: 95% (794/835) 2024-08-20T21:41:53.1941586Z remote: Compressing objects: 96% (802/835) 2024-08-20T21:41:53.1942120Z remote: Compressing objects: 97% (810/835) 2024-08-20T21:41:53.1942641Z remote: Compressing objects: 98% (819/835) 2024-08-20T21:41:53.1943182Z remote: Compressing objects: 99% (827/835) 2024-08-20T21:41:53.1945465Z remote: Compressing objects: 100% (835/835) 2024-08-20T21:41:53.1946047Z remote: Compressing objects: 100% (835/835), done. 2024-08-20T21:42:13.2065305Z remote: Total 1010152 (delta 1073), reused 1263 (delta 804), pack-reused 1008510 (from 1) 2024-08-20T21:42:34.2772446Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T21:42:34.2799684Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:42:34.2802549Z ##[endgroup] 2024-08-20T21:42:34.2803071Z ##[group]Determining the checkout info 2024-08-20T21:42:34.2803892Z ##[endgroup] 2024-08-20T21:42:34.2806188Z ##[group]Checking out the ref 2024-08-20T21:42:34.2807217Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:42:35.5919764Z ##[endgroup] 2024-08-20T21:42:35.5920382Z ##[group]Setting up auth for fetching submodules 2024-08-20T21:42:35.5921493Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T21:42:35.5967417Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T21:42:35.5994480Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T21:42:35.6022729Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T21:42:35.6047523Z ##[endgroup] 2024-08-20T21:42:35.6048028Z ##[group]Fetching submodules 2024-08-20T21:42:35.6051005Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T21:42:35.6340639Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T21:42:35.6617731Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T21:42:35.6619427Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T21:42:35.6620921Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T21:42:35.6622367Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T21:42:35.6626440Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T21:42:35.6628266Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T21:42:35.6629691Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T21:42:35.6632332Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T21:42:35.6634932Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T21:42:35.6637870Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T21:42:35.6640680Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T21:42:35.6643830Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T21:42:35.6647153Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T21:42:35.6650487Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T21:42:35.6654343Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T21:42:35.6658039Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:42:35.6663009Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T21:42:35.6666736Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T21:42:35.6670234Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T21:42:35.6674164Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T21:42:35.6678016Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T21:42:35.6681985Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T21:42:35.6685974Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T21:42:35.6690095Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T21:42:35.6694215Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T21:42:35.6699031Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T21:42:35.6703246Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T21:42:35.6707774Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T21:42:35.6712303Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T21:42:35.6717099Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T21:42:35.6721712Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T21:42:35.6726735Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T21:42:35.6731576Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T21:42:35.6738085Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T21:42:35.6765009Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T21:42:35.9882126Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T21:42:36.1653283Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T21:42:36.3417268Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T21:42:36.5750680Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T21:42:38.2147539Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T21:42:46.9046926Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T21:42:47.2614420Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T21:42:47.5921091Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T21:42:48.1369213Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T21:42:49.2185618Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T21:42:50.9827323Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T21:42:55.8083012Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T21:42:56.9704579Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T21:42:58.4670833Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T21:42:59.4770056Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T21:42:59.8657396Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T21:43:00.1468441Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T21:43:00.9980319Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T21:43:01.3064131Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T21:43:01.5817309Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T21:43:03.0403330Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T21:43:03.6446357Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T21:43:04.3062723Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T21:43:10.6002040Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T21:43:12.1610240Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T21:43:15.1049806Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T21:43:15.2920793Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T21:43:22.7925940Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T21:43:22.9621643Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T21:43:23.1418740Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T21:43:23.8825249Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T21:43:24.1536031Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T21:43:24.7779423Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T21:43:25.1753938Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T21:43:25.1865290Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T21:43:25.1953105Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T21:43:25.2177264Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T21:43:25.2526388Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T21:43:26.1364788Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T21:43:26.1579960Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T21:43:26.1997084Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T21:43:26.2889862Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T21:43:26.3196928Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T21:43:26.7815690Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T21:43:27.0150171Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T21:43:27.0902938Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T21:43:27.0919485Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:43:27.0922740Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:43:27.0924361Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:43:27.0925989Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:43:27.0927779Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:43:27.0953202Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T21:43:28.0489820Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T21:43:28.6378689Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T21:43:30.3820168Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T21:43:31.2477924Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T21:43:31.5778158Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T21:43:31.6669005Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T21:43:32.0376117Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T21:43:32.0950199Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T21:43:32.1065043Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T21:43:32.2131379Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T21:43:32.2505965Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T21:43:32.2867769Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T21:43:32.3098986Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T21:43:32.3520069Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T21:43:32.3642694Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T21:43:32.3660069Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T21:43:32.3683498Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T21:43:43.1522830Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T21:43:43.1692453Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T21:43:43.2502391Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T21:43:43.2519864Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:43:43.2522329Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:43:43.2524185Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:43:43.2547501Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T21:43:43.7267212Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T21:43:44.6807371Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T21:43:45.6237000Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T21:43:45.6251369Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:43:45.6253392Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:43:45.6255856Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:43:45.6258527Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:43:45.6261210Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:43:45.6263969Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:43:45.6266689Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:43:45.6269536Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:43:45.6296957Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T21:43:46.4158526Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T21:43:46.7535812Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T21:43:47.7456020Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T21:43:48.0005981Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T21:43:48.4749549Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T21:43:49.3617407Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T21:43:54.3749920Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T21:43:54.7514538Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T21:43:54.7691851Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T21:43:54.8038004Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T21:43:54.8164865Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T21:43:54.8179774Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:43:54.8206170Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T21:43:55.1011554Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T21:43:55.1183458Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T21:43:55.1548229Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T21:43:55.2496422Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T21:43:55.2649852Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T21:43:55.3028175Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T21:43:55.3558540Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T21:43:55.3898685Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T21:43:55.4116041Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T21:43:55.5074125Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T21:43:55.8307752Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T21:43:55.8341043Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T21:43:55.8342733Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T21:43:55.8370569Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T21:43:56.2337430Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T21:43:57.0294814Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T21:43:57.0605030Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T21:43:57.1225700Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T21:43:57.1244198Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:43:57.1246063Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:43:57.1248048Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:43:57.1250591Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:43:57.1253314Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:43:57.1255825Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:43:57.1258600Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:43:57.1261304Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:43:57.1287685Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T21:43:57.5181224Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T21:43:58.3917888Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T21:43:58.6714781Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T21:44:04.7268586Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T21:44:05.0195702Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T21:44:05.2477461Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T21:44:05.5364839Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T21:44:11.0946354Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T21:44:11.1333805Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T21:44:11.1479477Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T21:44:11.2396629Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T21:44:11.2515324Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T21:44:11.2647923Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T21:44:11.2789144Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T21:44:11.2805305Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:11.2808147Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:11.2828673Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T21:44:12.8018164Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T21:44:13.8983227Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T21:44:13.9415480Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T21:44:14.3702008Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T21:44:14.3812465Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T21:44:14.6245267Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T21:44:14.6263667Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:14.6265890Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:14.6293908Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T21:44:15.0736810Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T21:44:15.9305029Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T21:44:15.9966227Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T21:44:16.0048885Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T21:44:16.0163966Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T21:44:16.0500578Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T21:44:16.0769270Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T21:44:16.1146690Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T21:44:16.1384983Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T21:44:16.1402425Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:16.1404326Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:16.1406227Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:16.1407969Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:16.1433776Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T21:44:16.9870756Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T21:44:17.2108831Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T21:44:18.2627539Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T21:44:19.0808471Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T21:44:19.0944790Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T21:44:19.1567629Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T21:44:19.1824218Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T21:44:19.1841008Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:19.1863001Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T21:44:19.3961386Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T21:44:19.3999366Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T21:44:19.4314942Z Entering 'android/libs/fbjni' 2024-08-20T21:44:19.4358079Z Entering 'third_party/FP16' 2024-08-20T21:44:19.4398321Z Entering 'third_party/FXdiv' 2024-08-20T21:44:19.4438846Z Entering 'third_party/NNPACK' 2024-08-20T21:44:19.4481158Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:44:19.4521099Z Entering 'third_party/XNNPACK' 2024-08-20T21:44:19.4578949Z Entering 'third_party/benchmark' 2024-08-20T21:44:19.4619658Z Entering 'third_party/cpp-httplib' 2024-08-20T21:44:19.4659994Z Entering 'third_party/cpuinfo' 2024-08-20T21:44:19.4700028Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:44:19.4740036Z Entering 'third_party/cutlass' 2024-08-20T21:44:19.4788201Z Entering 'third_party/eigen' 2024-08-20T21:44:19.4828477Z Entering 'third_party/fbgemm' 2024-08-20T21:44:19.4867544Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:44:19.4904684Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:44:19.4942053Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:44:19.4986579Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:44:19.5024797Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:44:19.5063388Z Entering 'third_party/flatbuffers' 2024-08-20T21:44:19.5109109Z Entering 'third_party/fmt' 2024-08-20T21:44:19.5146440Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:44:19.5185652Z Entering 'third_party/gloo' 2024-08-20T21:44:19.5225061Z Entering 'third_party/googletest' 2024-08-20T21:44:19.5264636Z Entering 'third_party/ideep' 2024-08-20T21:44:19.5302466Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:44:19.5348420Z Entering 'third_party/ittapi' 2024-08-20T21:44:19.5387754Z Entering 'third_party/kineto' 2024-08-20T21:44:19.5427907Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:44:19.5467223Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:44:19.5506811Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:44:19.5546689Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:44:19.5584651Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:44:19.5622725Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:19.5663963Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:44:19.5702124Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:44:19.5741130Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:44:19.5780972Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:44:19.5821323Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:44:19.5860246Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:44:19.5907382Z Entering 'third_party/mimalloc' 2024-08-20T21:44:19.5946723Z Entering 'third_party/nccl/nccl' 2024-08-20T21:44:19.5985130Z Entering 'third_party/nlohmann' 2024-08-20T21:44:19.6024813Z Entering 'third_party/onnx' 2024-08-20T21:44:19.6079832Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:19.6119211Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:19.6160935Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:44:19.6200972Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:19.6238127Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:19.6275723Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:19.6314382Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:19.6352664Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:19.6389713Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:19.6427653Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:19.6463910Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:19.6510374Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:19.6549652Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:19.6609732Z Entering 'third_party/pocketfft' 2024-08-20T21:44:19.6648583Z Entering 'third_party/protobuf' 2024-08-20T21:44:19.6692147Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:19.6730138Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:19.6802850Z Entering 'third_party/psimd' 2024-08-20T21:44:19.6841591Z Entering 'third_party/pthreadpool' 2024-08-20T21:44:19.6879965Z Entering 'third_party/pybind11' 2024-08-20T21:44:19.6919617Z Entering 'third_party/python-peachpy' 2024-08-20T21:44:19.6959298Z Entering 'third_party/sleef' 2024-08-20T21:44:19.6997514Z Entering 'third_party/tensorpipe' 2024-08-20T21:44:19.7061507Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:19.7100688Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:19.7138005Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:19.7180056Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:19.7218160Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:19.7275418Z ##[endgroup] 2024-08-20T21:44:19.7276116Z ##[group]Persisting credentials for submodules 2024-08-20T21:44:19.7278639Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T21:44:19.7559866Z Entering 'android/libs/fbjni' 2024-08-20T21:44:19.7612246Z Entering 'third_party/FP16' 2024-08-20T21:44:19.7660144Z Entering 'third_party/FXdiv' 2024-08-20T21:44:19.7709361Z Entering 'third_party/NNPACK' 2024-08-20T21:44:19.7760598Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:44:19.7811008Z Entering 'third_party/XNNPACK' 2024-08-20T21:44:19.7878384Z Entering 'third_party/benchmark' 2024-08-20T21:44:19.7939334Z Entering 'third_party/cpp-httplib' 2024-08-20T21:44:19.7992351Z Entering 'third_party/cpuinfo' 2024-08-20T21:44:19.8042942Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:44:19.8093732Z Entering 'third_party/cutlass' 2024-08-20T21:44:19.8149786Z Entering 'third_party/eigen' 2024-08-20T21:44:19.8202367Z Entering 'third_party/fbgemm' 2024-08-20T21:44:19.8251424Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:44:19.8306500Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:44:19.8355132Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:44:19.8411177Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:44:19.8460568Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:44:19.8509679Z Entering 'third_party/flatbuffers' 2024-08-20T21:44:19.8565491Z Entering 'third_party/fmt' 2024-08-20T21:44:19.8616607Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:44:19.8664755Z Entering 'third_party/gloo' 2024-08-20T21:44:19.8715335Z Entering 'third_party/googletest' 2024-08-20T21:44:19.8766062Z Entering 'third_party/ideep' 2024-08-20T21:44:19.8814340Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:44:19.8867844Z Entering 'third_party/ittapi' 2024-08-20T21:44:19.8918795Z Entering 'third_party/kineto' 2024-08-20T21:44:19.8969958Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:44:19.9018515Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:44:19.9068666Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:44:19.9118831Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:44:19.9169010Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:44:19.9220226Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:19.9269176Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:44:19.9319743Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:44:19.9369011Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:44:19.9419464Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:44:19.9469001Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:44:19.9518178Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:44:19.9569528Z Entering 'third_party/mimalloc' 2024-08-20T21:44:19.9620804Z Entering 'third_party/nccl/nccl' 2024-08-20T21:44:19.9675211Z Entering 'third_party/nlohmann' 2024-08-20T21:44:19.9725202Z Entering 'third_party/onnx' 2024-08-20T21:44:19.9788100Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:19.9837879Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:19.9892228Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:44:19.9940945Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:19.9989660Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:20.0038878Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:20.0090724Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:20.0148579Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:20.0198095Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:20.0245657Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:20.0294339Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:20.0343849Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:20.0402256Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:20.0472104Z Entering 'third_party/pocketfft' 2024-08-20T21:44:20.0521675Z Entering 'third_party/protobuf' 2024-08-20T21:44:20.0578783Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:20.0639117Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:20.0694337Z Entering 'third_party/psimd' 2024-08-20T21:44:20.0746255Z Entering 'third_party/pthreadpool' 2024-08-20T21:44:20.0796999Z Entering 'third_party/pybind11' 2024-08-20T21:44:20.0849583Z Entering 'third_party/python-peachpy' 2024-08-20T21:44:20.0902880Z Entering 'third_party/sleef' 2024-08-20T21:44:20.0952755Z Entering 'third_party/tensorpipe' 2024-08-20T21:44:20.1002505Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:20.1049904Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:20.1106877Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:20.1154988Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:20.1203332Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:20.1272282Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T21:44:20.1550283Z Entering 'android/libs/fbjni' 2024-08-20T21:44:20.1597800Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T21:44:20.1613109Z Entering 'third_party/FP16' 2024-08-20T21:44:20.1660573Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T21:44:20.1677222Z Entering 'third_party/FXdiv' 2024-08-20T21:44:20.1724218Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T21:44:20.1739542Z Entering 'third_party/NNPACK' 2024-08-20T21:44:20.1788545Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T21:44:20.1804882Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:44:20.1848186Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T21:44:20.1862407Z Entering 'third_party/XNNPACK' 2024-08-20T21:44:20.1907142Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T21:44:20.1939472Z Entering 'third_party/benchmark' 2024-08-20T21:44:20.1984167Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:44:20.2001812Z Entering 'third_party/cpp-httplib' 2024-08-20T21:44:20.2045705Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T21:44:20.2060116Z Entering 'third_party/cpuinfo' 2024-08-20T21:44:20.2108607Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T21:44:20.2124960Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:44:20.2172668Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T21:44:20.2187404Z Entering 'third_party/cutlass' 2024-08-20T21:44:20.2235675Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T21:44:20.2258608Z Entering 'third_party/eigen' 2024-08-20T21:44:20.2301814Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T21:44:20.2322623Z Entering 'third_party/fbgemm' 2024-08-20T21:44:20.2372526Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T21:44:20.2385716Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:44:20.2433273Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T21:44:20.2447628Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:44:20.2492630Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T21:44:20.2506373Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:44:20.2553586Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T21:44:20.2575276Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:44:20.2621193Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:20.2637221Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:44:20.2684063Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T21:44:20.2700078Z Entering 'third_party/flatbuffers' 2024-08-20T21:44:20.2744705Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T21:44:20.2764266Z Entering 'third_party/fmt' 2024-08-20T21:44:20.2809619Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T21:44:20.2823147Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:44:20.2869064Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T21:44:20.2886013Z Entering 'third_party/gloo' 2024-08-20T21:44:20.2931077Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T21:44:20.2945184Z Entering 'third_party/googletest' 2024-08-20T21:44:20.2989859Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:20.3007403Z Entering 'third_party/ideep' 2024-08-20T21:44:20.3055393Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T21:44:20.3070067Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:44:20.3115691Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T21:44:20.3139201Z Entering 'third_party/ittapi' 2024-08-20T21:44:20.3182275Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T21:44:20.3198609Z Entering 'third_party/kineto' 2024-08-20T21:44:20.3244768Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T21:44:20.3259695Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:44:20.3305340Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T21:44:20.3321603Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:44:20.3368293Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T21:44:20.3383139Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:44:20.3431160Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T21:44:20.3445627Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:44:20.3497526Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T21:44:20.3513637Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:44:20.3561454Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T21:44:20.3574788Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:20.3621782Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T21:44:20.3639554Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:44:20.3686250Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T21:44:20.3700362Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:44:20.3745307Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:20.3761226Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:44:20.3808403Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T21:44:20.3821515Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:44:20.3867732Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T21:44:20.3885617Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:44:20.3930799Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T21:44:20.3943819Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:44:20.3990785Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T21:44:20.4007979Z Entering 'third_party/mimalloc' 2024-08-20T21:44:20.4054287Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T21:44:20.4067878Z Entering 'third_party/nccl/nccl' 2024-08-20T21:44:20.4116867Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T21:44:20.4132874Z Entering 'third_party/nlohmann' 2024-08-20T21:44:20.4177997Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T21:44:20.4195588Z Entering 'third_party/onnx' 2024-08-20T21:44:20.4242394Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T21:44:20.4274282Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:20.4319745Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:44:20.4334971Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:20.4380537Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:44:20.4399199Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:44:20.4447549Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T21:44:20.4463001Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:20.4509277Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:44:20.4524508Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:20.4571079Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:20.4587337Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:20.4633609Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T21:44:20.4646831Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:20.4691635Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T21:44:20.4706776Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:20.4753293Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T21:44:20.4768507Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:20.4816541Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T21:44:20.4829654Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:20.4879130Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T21:44:20.4892305Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:20.4939835Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T21:44:20.4957265Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:20.5003546Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T21:44:20.5019567Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:20.5069238Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T21:44:20.5100858Z Entering 'third_party/pocketfft' 2024-08-20T21:44:20.5147426Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T21:44:20.5164133Z Entering 'third_party/protobuf' 2024-08-20T21:44:20.5211804Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T21:44:20.5228120Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:20.5275581Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:44:20.5290749Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:20.5337422Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:20.5352616Z Entering 'third_party/psimd' 2024-08-20T21:44:20.5402114Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T21:44:20.5416674Z Entering 'third_party/pthreadpool' 2024-08-20T21:44:20.5461352Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T21:44:20.5479792Z Entering 'third_party/pybind11' 2024-08-20T21:44:20.5529607Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:44:20.5543093Z Entering 'third_party/python-peachpy' 2024-08-20T21:44:20.5587979Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T21:44:20.5604221Z Entering 'third_party/sleef' 2024-08-20T21:44:20.5648280Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T21:44:20.5662675Z Entering 'third_party/tensorpipe' 2024-08-20T21:44:20.5709946Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T21:44:20.5726929Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:20.5772143Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:44:20.5785203Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:20.5830575Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T21:44:20.5845802Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:20.5892480Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T21:44:20.5905350Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:20.5950672Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:44:20.5966328Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:20.6012467Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T21:44:20.6810962Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T21:44:20.7096777Z Entering 'android/libs/fbjni' 2024-08-20T21:44:20.7135705Z Entering 'third_party/FP16' 2024-08-20T21:44:20.7174656Z Entering 'third_party/FXdiv' 2024-08-20T21:44:20.7215136Z Entering 'third_party/NNPACK' 2024-08-20T21:44:20.7261500Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:44:20.7300117Z Entering 'third_party/XNNPACK' 2024-08-20T21:44:20.7354264Z Entering 'third_party/benchmark' 2024-08-20T21:44:20.7394474Z Entering 'third_party/cpp-httplib' 2024-08-20T21:44:20.7439140Z Entering 'third_party/cpuinfo' 2024-08-20T21:44:20.7478682Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:44:20.7516880Z Entering 'third_party/cutlass' 2024-08-20T21:44:20.7563751Z Entering 'third_party/eigen' 2024-08-20T21:44:20.7605042Z Entering 'third_party/fbgemm' 2024-08-20T21:44:20.7643858Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:44:20.7682288Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:44:20.7720198Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:44:20.7763902Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:44:20.7801636Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:44:20.7840217Z Entering 'third_party/flatbuffers' 2024-08-20T21:44:20.7881665Z Entering 'third_party/fmt' 2024-08-20T21:44:20.7919734Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:44:20.7959516Z Entering 'third_party/gloo' 2024-08-20T21:44:20.7997983Z Entering 'third_party/googletest' 2024-08-20T21:44:20.8036694Z Entering 'third_party/ideep' 2024-08-20T21:44:20.8074652Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:44:20.8127352Z Entering 'third_party/ittapi' 2024-08-20T21:44:20.8167409Z Entering 'third_party/kineto' 2024-08-20T21:44:20.8207219Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:44:20.8244361Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:44:20.8283581Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:44:20.8321315Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:44:20.8360100Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:44:20.8397719Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:20.8437910Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:44:20.8477407Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:44:20.8515501Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:44:20.8553408Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:44:20.8598298Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:44:20.8637413Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:44:20.8678289Z Entering 'third_party/mimalloc' 2024-08-20T21:44:20.8716635Z Entering 'third_party/nccl/nccl' 2024-08-20T21:44:20.8754845Z Entering 'third_party/nlohmann' 2024-08-20T21:44:20.8797446Z Entering 'third_party/onnx' 2024-08-20T21:44:20.8849414Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:20.8889939Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:20.8931896Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:44:20.8980067Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:20.9017137Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:20.9056872Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:20.9094524Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:20.9139329Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:20.9177452Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:20.9214699Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:20.9256581Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:20.9299235Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:20.9339228Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:20.9398683Z Entering 'third_party/pocketfft' 2024-08-20T21:44:20.9436553Z Entering 'third_party/protobuf' 2024-08-20T21:44:20.9478039Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:20.9515559Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:20.9553224Z Entering 'third_party/psimd' 2024-08-20T21:44:20.9593697Z Entering 'third_party/pthreadpool' 2024-08-20T21:44:20.9637044Z Entering 'third_party/pybind11' 2024-08-20T21:44:20.9676913Z Entering 'third_party/python-peachpy' 2024-08-20T21:44:20.9714115Z Entering 'third_party/sleef' 2024-08-20T21:44:20.9752636Z Entering 'third_party/tensorpipe' 2024-08-20T21:44:20.9794304Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:20.9830170Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:20.9867003Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:20.9907868Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:20.9945634Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:20.9998748Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T21:44:21.0279925Z Entering 'android/libs/fbjni' 2024-08-20T21:44:21.0316447Z Entering 'third_party/FP16' 2024-08-20T21:44:21.0353455Z Entering 'third_party/FXdiv' 2024-08-20T21:44:21.0394304Z Entering 'third_party/NNPACK' 2024-08-20T21:44:21.0434922Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:44:21.0475616Z Entering 'third_party/XNNPACK' 2024-08-20T21:44:21.0533957Z Entering 'third_party/benchmark' 2024-08-20T21:44:21.0575281Z Entering 'third_party/cpp-httplib' 2024-08-20T21:44:21.0614710Z Entering 'third_party/cpuinfo' 2024-08-20T21:44:21.0653205Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:44:21.0693945Z Entering 'third_party/cutlass' 2024-08-20T21:44:21.0743303Z Entering 'third_party/eigen' 2024-08-20T21:44:21.0789781Z Entering 'third_party/fbgemm' 2024-08-20T21:44:21.0828449Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:44:21.0869994Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:44:21.0906116Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:44:21.0949953Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:44:21.0987080Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:44:21.1030291Z Entering 'third_party/flatbuffers' 2024-08-20T21:44:21.1069999Z Entering 'third_party/fmt' 2024-08-20T21:44:21.1107942Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:44:21.1147024Z Entering 'third_party/gloo' 2024-08-20T21:44:21.1187953Z Entering 'third_party/googletest' 2024-08-20T21:44:21.1226241Z Entering 'third_party/ideep' 2024-08-20T21:44:21.1262476Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:44:21.1318089Z Entering 'third_party/ittapi' 2024-08-20T21:44:21.1361054Z Entering 'third_party/kineto' 2024-08-20T21:44:21.1397760Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:44:21.1435256Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:44:21.1475592Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:44:21.1514473Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:44:21.1551029Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:44:21.1588244Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:44:21.1634276Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:44:21.1674266Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:44:21.1712570Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:44:21.1752236Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:44:21.1793094Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:44:21.1830001Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:44:21.1868229Z Entering 'third_party/mimalloc' 2024-08-20T21:44:21.1906462Z Entering 'third_party/nccl/nccl' 2024-08-20T21:44:21.1944413Z Entering 'third_party/nlohmann' 2024-08-20T21:44:21.1983193Z Entering 'third_party/onnx' 2024-08-20T21:44:21.2037305Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:44:21.2075259Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:44:21.2115978Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:44:21.2157585Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:44:21.2196808Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:44:21.2235189Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:44:21.2277926Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:44:21.2317110Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:44:21.2352930Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:44:21.2391866Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:44:21.2461545Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:44:21.2502908Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:44:21.2549353Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:44:21.2607972Z Entering 'third_party/pocketfft' 2024-08-20T21:44:21.2645587Z Entering 'third_party/protobuf' 2024-08-20T21:44:21.2687047Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:44:21.2723878Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:44:21.2763988Z Entering 'third_party/psimd' 2024-08-20T21:44:21.2802027Z Entering 'third_party/pthreadpool' 2024-08-20T21:44:21.2840013Z Entering 'third_party/pybind11' 2024-08-20T21:44:21.2878321Z Entering 'third_party/python-peachpy' 2024-08-20T21:44:21.2917030Z Entering 'third_party/sleef' 2024-08-20T21:44:21.2953674Z Entering 'third_party/tensorpipe' 2024-08-20T21:44:21.2993356Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:44:21.3039298Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:44:21.3077166Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:44:21.3115212Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:44:21.3150926Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:44:21.3206881Z ##[endgroup] 2024-08-20T21:44:21.3241703Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T21:44:21.3267883Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T21:44:21.3460785Z Prepare all required actions 2024-08-20T21:44:21.3461242Z Getting action download info 2024-08-20T21:44:21.5444994Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T21:44:21.5445428Z env: 2024-08-20T21:44:21.5445705Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:21.5446049Z ##[endgroup] 2024-08-20T21:44:21.5505300Z ##[group]Run set -euo pipefail 2024-08-20T21:44:21.5505932Z set -euo pipefail 2024-08-20T21:44:21.5506330Z function get_ec2_metadata() { 2024-08-20T21:44:21.5506878Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T21:44:21.5507817Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T21:44:21.5508631Z  category=$1 2024-08-20T21:44:21.5509150Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T21:44:21.5509806Z  runner_name_str=i-0f00d0841bbe75141 2024-08-20T21:44:21.5510336Z  if [[ -f /.inarc ]]; then 2024-08-20T21:44:21.5510819Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T21:44:21.5511495Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T21:44:21.5512156Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T21:44:21.5512737Z  else 2024-08-20T21:44:21.5513182Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T21:44:21.5513728Z  fi 2024-08-20T21:44:21.5513987Z } 2024-08-20T21:44:21.5514320Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T21:44:21.5514861Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T21:44:21.5515489Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T21:44:21.5516028Z echo "system info $(uname -a)" 2024-08-20T21:44:21.5523819Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:21.5524281Z env: 2024-08-20T21:44:21.5524552Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:21.5524885Z ##[endgroup] 2024-08-20T21:44:21.5624201Z ami-id: ami-06c68f701d8090592 2024-08-20T21:44:21.5682423Z instance-id: i-0f00d0841bbe75141 2024-08-20T21:44:21.5734428Z instance-type: c5.12xlarge 2024-08-20T21:44:21.5746143Z system info Linux ip-10-0-13-192.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T21:44:21.5765705Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T21:44:21.5766584Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T21:44:21.5772465Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:21.5772917Z env: 2024-08-20T21:44:21.5773162Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:21.5773478Z ##[endgroup] 2024-08-20T21:44:21.5864336Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T21:44:21.5864966Z if systemctl is-active --quiet docker; then 2024-08-20T21:44:21.5865516Z  echo "Docker daemon is running..."; 2024-08-20T21:44:21.5865968Z else 2024-08-20T21:44:21.5866447Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T21:44:21.5867046Z fi 2024-08-20T21:44:21.5872525Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:21.5873000Z env: 2024-08-20T21:44:21.5873254Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:21.5873571Z ##[endgroup] 2024-08-20T21:44:21.5947414Z Docker daemon is running... 2024-08-20T21:44:21.6010890Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T21:44:21.6011421Z with: 2024-08-20T21:44:21.6011659Z shell: bash 2024-08-20T21:44:21.6011911Z timeout_minutes: 5 2024-08-20T21:44:21.6012203Z max_attempts: 3 2024-08-20T21:44:21.6012488Z retry_wait_seconds: 30 2024-08-20T21:44:21.6015649Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T21:44:21.6019162Z polling_interval_seconds: 1 2024-08-20T21:44:21.6019594Z warning_on_retry: true 2024-08-20T21:44:21.6019923Z continue_on_error: false 2024-08-20T21:44:21.6020251Z env: 2024-08-20T21:44:21.6020506Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:21.6020852Z AWS_RETRY_MODE: standard 2024-08-20T21:44:21.6021174Z AWS_MAX_ATTEMPTS: 5 2024-08-20T21:44:21.6021505Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T21:44:21.6021877Z ##[endgroup] 2024-08-20T21:44:22.6790596Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:44:22.6791457Z Configure a credential helper to remove this warning. See 2024-08-20T21:44:22.6792344Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:44:22.6792911Z 2024-08-20T21:44:22.6793026Z Login Succeeded 2024-08-20T21:44:23.6545274Z Command completed after 1 attempt(s). 2024-08-20T21:44:23.6603311Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:44:23.6604062Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:44:23.6604740Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:44:23.6611237Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:23.6611711Z env: 2024-08-20T21:44:23.6611974Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:23.6612305Z ##[endgroup] 2024-08-20T21:44:23.6702888Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T21:44:23.6703692Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T21:44:23.6704309Z # shellcheck disable=SC2046 2024-08-20T21:44:23.6704776Z docker stop $(docker ps -q) || true 2024-08-20T21:44:23.6705245Z # Prune all of the docker images 2024-08-20T21:44:23.6705706Z docker system prune -af 2024-08-20T21:44:23.6711732Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:23.6712208Z env: 2024-08-20T21:44:23.6712450Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:23.6712779Z ##[endgroup] 2024-08-20T21:44:23.7001735Z "docker stop" requires at least 1 argument. 2024-08-20T21:44:23.7002493Z See 'docker stop --help'. 2024-08-20T21:44:23.7002732Z 2024-08-20T21:44:23.7002968Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T21:44:23.7003347Z 2024-08-20T21:44:23.7005626Z Stop one or more running containers 2024-08-20T21:44:23.7179174Z Total reclaimed space: 0B 2024-08-20T21:44:23.7216120Z ##[group]Run set +e 2024-08-20T21:44:23.7216452Z set +e 2024-08-20T21:44:23.7216740Z set -x 2024-08-20T21:44:23.7217013Z  2024-08-20T21:44:23.7217298Z PT_DOMAIN=download.pytorch.org 2024-08-20T21:44:23.7218091Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T21:44:23.7219593Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T21:44:23.7220370Z # one is returned at random 2024-08-20T21:44:23.7220913Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T21:44:23.7221416Z  2024-08-20T21:44:23.7221717Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T21:44:23.7222338Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T21:44:23.7223268Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T21:44:23.7223819Z  2024-08-20T21:44:23.7224128Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T21:44:23.7224676Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T21:44:23.7225190Z  exit 1 2024-08-20T21:44:23.7225476Z  fi 2024-08-20T21:44:23.7225842Z fi 2024-08-20T21:44:23.7226106Z  2024-08-20T21:44:23.7226451Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T21:44:23.7226953Z  # Clean up any old records first 2024-08-20T21:44:23.7227462Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T21:44:23.7227914Z fi 2024-08-20T21:44:23.7228159Z  2024-08-20T21:44:23.7228578Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T21:44:23.7229127Z cat /etc/hosts 2024-08-20T21:44:23.7234624Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:23.7235071Z env: 2024-08-20T21:44:23.7235328Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:23.7235646Z ##[endgroup] 2024-08-20T21:44:23.7257256Z + PT_DOMAIN=download.pytorch.org 2024-08-20T21:44:23.7288818Z ++ dig -4 +short download.pytorch.org 2024-08-20T21:44:23.7289273Z ++ tail -n1 2024-08-20T21:44:23.7651584Z + RESOLVED_IP=108.138.64.116 2024-08-20T21:44:23.7652280Z + '[' -z 108.138.64.116 ']' 2024-08-20T21:44:23.7652832Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T21:44:23.7665432Z + echo '108.138.64.116 download.pytorch.org' 2024-08-20T21:44:23.7665972Z + sudo tee -a /etc/hosts 2024-08-20T21:44:23.9966043Z 108.138.64.116 download.pytorch.org 2024-08-20T21:44:23.9979743Z + cat /etc/hosts 2024-08-20T21:44:23.9986906Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T21:44:23.9998031Z ::1 localhost6 localhost6.localdomain6 2024-08-20T21:44:23.9998660Z 108.138.64.116 download.pytorch.org 2024-08-20T21:44:24.0159144Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T21:44:24.0159762Z with: 2024-08-20T21:44:24.0160827Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.0162044Z docker-build-dir: .ci/docker 2024-08-20T21:44:24.0162415Z working-directory: . 2024-08-20T21:44:24.0162890Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:24.0163511Z force-push: false 2024-08-20T21:44:24.0163786Z env: 2024-08-20T21:44:24.0164032Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:24.0164340Z ##[endgroup] 2024-08-20T21:44:24.0181823Z ##[group]Run set -ex 2024-08-20T21:44:24.0182192Z set -ex 2024-08-20T21:44:24.0182456Z  2024-08-20T21:44:24.0183016Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T21:44:24.0184063Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T21:44:24.0184879Z # job could then download the pre-built image as usual 2024-08-20T21:44:24.0185631Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T21:44:24.0186316Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:24.0186955Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:24.0187527Z  2024-08-20T21:44:24.0188021Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T21:44:24.0188652Z  exit 0 2024-08-20T21:44:24.0188934Z else 2024-08-20T21:44:24.0189271Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:24.0189712Z fi 2024-08-20T21:44:24.0189965Z  2024-08-20T21:44:24.0190420Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T21:44:24.0191359Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T21:44:24.0192099Z  # use it as it is, but first let's extract the tag 2024-08-20T21:44:24.0192759Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T21:44:24.0193439Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:24.0194227Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:24.0194734Z else 2024-08-20T21:44:24.0195125Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T21:44:24.0195748Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:24.0196605Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:24.0197310Z fi 2024-08-20T21:44:24.0204051Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:24.0204517Z env: 2024-08-20T21:44:24.0204757Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:24.0205059Z REPO_NAME: pytorch 2024-08-20T21:44:24.0206084Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.0207187Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T21:44:24.0207653Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:24.0208119Z ##[endgroup] 2024-08-20T21:44:24.0231373Z + [[ ! -d .ci/docker ]] 2024-08-20T21:44:24.0231781Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T21:44:24.0232155Z + echo skip=false 2024-08-20T21:44:24.0234000Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T21:44:24.0239253Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.0240441Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T21:44:24.0260553Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.0261315Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.0262938Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.0287011Z ##[group]Run set +e 2024-08-20T21:44:24.0287343Z set +e 2024-08-20T21:44:24.0287606Z set -x 2024-08-20T21:44:24.0287846Z  2024-08-20T21:44:24.0288088Z login() { 2024-08-20T21:44:24.0288704Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T21:44:24.0289385Z } 2024-08-20T21:44:24.0289614Z  2024-08-20T21:44:24.0289854Z retry () { 2024-08-20T21:44:24.0290205Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T21:44:24.0290613Z } 2024-08-20T21:44:24.0290837Z  2024-08-20T21:44:24.0291114Z retry login "${DOCKER_REGISTRY}" 2024-08-20T21:44:24.0291495Z  2024-08-20T21:44:24.0291911Z # Check if image already exists, if it does then skip building it 2024-08-20T21:44:24.0292579Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T21:44:24.0293046Z  exit 0 2024-08-20T21:44:24.0293313Z fi 2024-08-20T21:44:24.0293534Z  2024-08-20T21:44:24.0293996Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T21:44:24.0294786Z # be empty. The default action would be to continue rebuild the image 2024-08-20T21:44:24.0295492Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T21:44:24.0296110Z  # if we're on the base branch then use the parent commit 2024-08-20T21:44:24.0296655Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T21:44:24.0297048Z else 2024-08-20T21:44:24.0297472Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T21:44:24.0298108Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T21:44:24.0298848Z fi 2024-08-20T21:44:24.0299254Z  2024-08-20T21:44:24.0299550Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T21:44:24.0300139Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:24.0300591Z  2024-08-20T21:44:24.0301234Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T21:44:24.0302020Z  exit 0 2024-08-20T21:44:24.0302299Z fi 2024-08-20T21:44:24.0302555Z  2024-08-20T21:44:24.0302969Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T21:44:24.0303975Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T21:44:24.0304807Z  exit 1 2024-08-20T21:44:24.0305089Z fi 2024-08-20T21:44:24.0305331Z  2024-08-20T21:44:24.0305808Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T21:44:24.0306800Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T21:44:24.0307693Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T21:44:24.0308695Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T21:44:24.0309825Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T21:44:24.0310483Z fi 2024-08-20T21:44:24.0310839Z  2024-08-20T21:44:24.0311266Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:44:24.0316279Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:24.0316735Z env: 2024-08-20T21:44:24.0316974Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:24.0317300Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T21:44:24.0317714Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:44:24.0318893Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.0320043Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.0320608Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:24.0321074Z ##[endgroup] 2024-08-20T21:44:24.0342815Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:24.0343687Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:24.0345109Z + aws ecr get-login-password --region us-east-1 2024-08-20T21:44:24.0346242Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:24.5415501Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:44:24.5416801Z Configure a credential helper to remove this warning. See 2024-08-20T21:44:24.5417809Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:44:24.5418517Z 2024-08-20T21:44:24.5418659Z Login Succeeded 2024-08-20T21:44:24.5429158Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.7412389Z { 2024-08-20T21:44:24.7412972Z "schemaVersion": 2, 2024-08-20T21:44:24.7413811Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T21:44:24.7414555Z "config": { 2024-08-20T21:44:24.7555932Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T21:44:24.7556569Z "size": 43916, 2024-08-20T21:44:24.7557182Z "digest": "sha256:822356886a68936cce34917f5ffc60d864918fab7594bb08c26bf1816dd6110d" 2024-08-20T21:44:24.7557832Z }, 2024-08-20T21:44:24.7558074Z "layers": [ 2024-08-20T21:44:24.7558377Z { 2024-08-20T21:44:24.7558885Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7559732Z "size": 30440714, 2024-08-20T21:44:24.7560295Z "digest": "sha256:762bedf4b1b784c3de6c5022c5307d63123d3b7cdd59211317e37e9d477deaa0" 2024-08-20T21:44:24.7560983Z }, 2024-08-20T21:44:24.7561228Z { 2024-08-20T21:44:24.7561671Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7562236Z "size": 1822, 2024-08-20T21:44:24.7562558Z + exit 0 2024-08-20T21:44:24.7563095Z "digest": "sha256:347beb78d39bc9079a3c20e891908ad5695cdea4a312b0e1de586110bb9ee72f" 2024-08-20T21:44:24.7563756Z }, 2024-08-20T21:44:24.7563981Z { 2024-08-20T21:44:24.7564432Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7565015Z "size": 313164873, 2024-08-20T21:44:24.7565572Z "digest": "sha256:71b614615bcb09bc43ad35a0238b1b82945122c70d95efa9728a6a044a7ddb73" 2024-08-20T21:44:24.7566221Z }, 2024-08-20T21:44:24.7566461Z { 2024-08-20T21:44:24.7566898Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7567469Z "size": 803, 2024-08-20T21:44:24.7568029Z "digest": "sha256:f96c8558662d14e2cc064f94a0166dbf827cf4b783c8d3edb4dbc19dcfff8ca3" 2024-08-20T21:44:24.7568672Z }, 2024-08-20T21:44:24.7568915Z { 2024-08-20T21:44:24.7569343Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7569919Z "size": 106, 2024-08-20T21:44:24.7570485Z "digest": "sha256:c81ff9d20d36fbd48a567918c08f020d23bd83515aec7d23fda14cd56786be81" 2024-08-20T21:44:24.7571136Z }, 2024-08-20T21:44:24.7571370Z { 2024-08-20T21:44:24.7571941Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7572614Z "size": 544, 2024-08-20T21:44:24.7573181Z "digest": "sha256:151e9a9d289467dccd8eefb461af7188b39953d6ac10b678133551bd20586ce6" 2024-08-20T21:44:24.7573841Z }, 2024-08-20T21:44:24.7574073Z { 2024-08-20T21:44:24.7574530Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7575109Z "size": 1214, 2024-08-20T21:44:24.7575682Z "digest": "sha256:8538dcd89f0cac92de23cde0e22b0e7cfde631fbef3062f6387bbc78c4b8c051" 2024-08-20T21:44:24.7576351Z }, 2024-08-20T21:44:24.7576600Z { 2024-08-20T21:44:24.7577048Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7577654Z "size": 483, 2024-08-20T21:44:24.7578224Z "digest": "sha256:cd13007f6f0922ba40ddf0f45d8693c116a5f27e894bd04a34e40eb57bd5c640" 2024-08-20T21:44:24.7579035Z }, 2024-08-20T21:44:24.7579283Z { 2024-08-20T21:44:24.7579726Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7580311Z "size": 110321925, 2024-08-20T21:44:24.7580902Z "digest": "sha256:74fd96b6ebd997dccc9767b9222e3138502d240bca371c2dab11ed193140d9cb" 2024-08-20T21:44:24.7581619Z }, 2024-08-20T21:44:24.7581849Z { 2024-08-20T21:44:24.7582292Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7582869Z "size": 3840, 2024-08-20T21:44:24.7583429Z "digest": "sha256:ce4e1800db98e945fe1b38aac9ac5afcc6edcf1c9cc321499bcb108d04fe1442" 2024-08-20T21:44:24.7584098Z }, 2024-08-20T21:44:24.7584345Z { 2024-08-20T21:44:24.7584786Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7585348Z "size": 1910, 2024-08-20T21:44:24.7585893Z "digest": "sha256:402b067f30bf9e459448062a2a55e42169b09275d1fa427073a332adc6929112" 2024-08-20T21:44:24.7586523Z }, 2024-08-20T21:44:24.7586764Z { 2024-08-20T21:44:24.7587188Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7587767Z "size": 701, 2024-08-20T21:44:24.7588345Z "digest": "sha256:2cebdfe4c5a8da099cc35bcc957374e8545fed39fbf41715974c4d4c1ec3efa9" 2024-08-20T21:44:24.7588992Z }, 2024-08-20T21:44:24.7589237Z { 2024-08-20T21:44:24.7589679Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7590254Z "size": 2663407704, 2024-08-20T21:44:24.7590837Z "digest": "sha256:c6e13ff8aa9a65d1d347f8a336df5ca7febab9d34a3027c2b3c138c1a711baf4" 2024-08-20T21:44:24.7591597Z }, 2024-08-20T21:44:24.7591839Z { 2024-08-20T21:44:24.7592279Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7592838Z "size": 32, 2024-08-20T21:44:24.7593403Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7594058Z }, 2024-08-20T21:44:24.7594283Z { 2024-08-20T21:44:24.7594725Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7595299Z "size": 379, 2024-08-20T21:44:24.7595882Z "digest": "sha256:e238df05ddcc0ec1e1ce769b25f7103f76af6aa35e4aa4ccd1adcdb7621158f6" 2024-08-20T21:44:24.7596526Z }, 2024-08-20T21:44:24.7596769Z { 2024-08-20T21:44:24.7597211Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7597789Z "size": 222887, 2024-08-20T21:44:24.7598329Z "digest": "sha256:a88675bdc72381c487323d232044cca10dd1f77e62eaf00a650bd28d27d6f9bf" 2024-08-20T21:44:24.7598970Z }, 2024-08-20T21:44:24.7599277Z { 2024-08-20T21:44:24.7599720Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7600275Z "size": 231, 2024-08-20T21:44:24.7600832Z "digest": "sha256:f4cb80c411d1b5fb90688374327599a7de6ea89c800473afb1b9766003d5a39a" 2024-08-20T21:44:24.7601470Z }, 2024-08-20T21:44:24.7601694Z { 2024-08-20T21:44:24.7602134Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7602712Z "size": 2933275, 2024-08-20T21:44:24.7603410Z "digest": "sha256:c9589ebd9ae736462b4dc9ea4763b2892eae668c19ca321e7c9ef2762b86a528" 2024-08-20T21:44:24.7604042Z }, 2024-08-20T21:44:24.7604291Z { 2024-08-20T21:44:24.7604737Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7605313Z "size": 1805, 2024-08-20T21:44:24.7605861Z "digest": "sha256:dd51c681db1409845c172ac4fc813dfc9db27bc797fe919a47724410acd481c4" 2024-08-20T21:44:24.7606510Z }, 2024-08-20T21:44:24.7606754Z { 2024-08-20T21:44:24.7607184Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7607754Z "size": 105, 2024-08-20T21:44:24.7608310Z "digest": "sha256:fc983e33a069ae2891a2161f9d24540f9e2728dddc9fd0e63c5b82454ab51f61" 2024-08-20T21:44:24.7608958Z }, 2024-08-20T21:44:24.7609183Z { 2024-08-20T21:44:24.7609622Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7610195Z "size": 564, 2024-08-20T21:44:24.7610752Z "digest": "sha256:bc705d688c776732ddd22c3540b0e135d7dc23e4d46b64918be11b2fa1a265c5" 2024-08-20T21:44:24.7611381Z }, 2024-08-20T21:44:24.7611623Z { 2024-08-20T21:44:24.7612059Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7612631Z "size": 32, 2024-08-20T21:44:24.7613170Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7613825Z }, 2024-08-20T21:44:24.7614068Z { 2024-08-20T21:44:24.7614493Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7615071Z "size": 104, 2024-08-20T21:44:24.7615620Z "digest": "sha256:99f6c1ac758215178233da5104ddb298caf3ae441129209a625e9e649463b980" 2024-08-20T21:44:24.7616260Z }, 2024-08-20T21:44:24.7616484Z { 2024-08-20T21:44:24.7616922Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7617497Z "size": 504, 2024-08-20T21:44:24.7618063Z "digest": "sha256:c409d27a5a9560bf329dec2ed58b701a43f1ffa8babc7ca13c61321cf22ef007" 2024-08-20T21:44:24.7618825Z }, 2024-08-20T21:44:24.7619073Z { 2024-08-20T21:44:24.7619522Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7620084Z "size": 101109909, 2024-08-20T21:44:24.7620676Z "digest": "sha256:a3a398ca1120cf9319dbbb81b12ddc41f7bf3b5bd3f3612c17c2bf7b5194e505" 2024-08-20T21:44:24.7621334Z }, 2024-08-20T21:44:24.7621574Z { 2024-08-20T21:44:24.7621999Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7622674Z "size": 109, 2024-08-20T21:44:24.7623224Z "digest": "sha256:314875f5fae452a321bd50b8527ebb8cf7db0a53ff30293472014390479fcf64" 2024-08-20T21:44:24.7623867Z }, 2024-08-20T21:44:24.7624091Z { 2024-08-20T21:44:24.7624535Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7625103Z "size": 490, 2024-08-20T21:44:24.7625667Z "digest": "sha256:8a5e90dafefa97ef114ac2d4c2f8e000536425318010f40d15bfdc6f8d7e2a79" 2024-08-20T21:44:24.7626302Z }, 2024-08-20T21:44:24.7626542Z { 2024-08-20T21:44:24.7626986Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7627536Z "size": 333, 2024-08-20T21:44:24.7628094Z "digest": "sha256:e3b946ab3378ddc5e96c6355bd0aaa135f29b1ea34b417652442c9df42b6eb49" 2024-08-20T21:44:24.7628740Z }, 2024-08-20T21:44:24.7628975Z { 2024-08-20T21:44:24.7629399Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7629972Z "size": 103, 2024-08-20T21:44:24.7630509Z "digest": "sha256:14a85a990f7189af056863e625917e6a5583ce8a7b1d319783011803ea1a9da4" 2024-08-20T21:44:24.7631134Z }, 2024-08-20T21:44:24.7631355Z { 2024-08-20T21:44:24.7631923Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7632496Z "size": 1473, 2024-08-20T21:44:24.7633037Z "digest": "sha256:16d809b167d49638350b86af396a05eca356ba903ca4953a083d29f28154c16a" 2024-08-20T21:44:24.7633652Z }, 2024-08-20T21:44:24.7633893Z { 2024-08-20T21:44:24.7634390Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7634945Z "size": 611525039, 2024-08-20T21:44:24.7635520Z "digest": "sha256:9b1a95eedb8338e08a9ba00b933ef64420d41664b1a4c2a40f1703c79a3d2ba1" 2024-08-20T21:44:24.7636158Z }, 2024-08-20T21:44:24.7636392Z { 2024-08-20T21:44:24.7636812Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7637381Z "size": 164, 2024-08-20T21:44:24.7637927Z "digest": "sha256:b35ca05427236370ed98f905cf9b8a0e4599173ef1843504fc2547e9525cd483" 2024-08-20T21:44:24.7638556Z }, 2024-08-20T21:44:24.7638779Z { 2024-08-20T21:44:24.7639208Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7639936Z "size": 2210, 2024-08-20T21:44:24.7640498Z "digest": "sha256:bad10ac3a6279679db06f845b321a9fdaef80be969acbfd257f9e807762953a8" 2024-08-20T21:44:24.7641133Z }, 2024-08-20T21:44:24.7641367Z { 2024-08-20T21:44:24.7641797Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7642350Z "size": 152, 2024-08-20T21:44:24.7642910Z "digest": "sha256:943da2164c7a7bf690bcbc751afe5ea646925ddf0ab96fd2ce48b0a52bd66e9d" 2024-08-20T21:44:24.7643721Z }, 2024-08-20T21:44:24.7643954Z { 2024-08-20T21:44:24.7644438Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7645008Z "size": 648, 2024-08-20T21:44:24.7645552Z "digest": "sha256:2d276441bd8ae528772f4fd876af429ab3518bebf61090795ad3b6aced9d60e4" 2024-08-20T21:44:24.7646195Z }, 2024-08-20T21:44:24.7646415Z { 2024-08-20T21:44:24.7646844Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7647407Z "size": 32, 2024-08-20T21:44:24.7647946Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7648591Z }, 2024-08-20T21:44:24.7648828Z { 2024-08-20T21:44:24.7649260Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7649812Z "size": 167, 2024-08-20T21:44:24.7650365Z "digest": "sha256:a3ef8d7d02be75588561825a22ecbdc3129ad8a543f4d1415fee8284d91327bb" 2024-08-20T21:44:24.7650998Z }, 2024-08-20T21:44:24.7651232Z { 2024-08-20T21:44:24.7651652Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7652222Z "size": 109, 2024-08-20T21:44:24.7652770Z "digest": "sha256:6aaed7a7874900b7539103d4ef762ab0f9a2d760c898f2ebcf135b73858aeed5" 2024-08-20T21:44:24.7653517Z }, 2024-08-20T21:44:24.7653739Z { 2024-08-20T21:44:24.7654172Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7654734Z "size": 102, 2024-08-20T21:44:24.7655262Z "digest": "sha256:944e780839169e63df50b5cc05cc9fb99e8ed0b9692d56cee9cd06137bdb08e3" 2024-08-20T21:44:24.7655890Z }, 2024-08-20T21:44:24.7656123Z { 2024-08-20T21:44:24.7656664Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7657201Z "size": 422, 2024-08-20T21:44:24.7657726Z "digest": "sha256:9602de36240c11e5eb9a1926c6443a2ff43948a958a15162e7aa827e1f83ad3e" 2024-08-20T21:44:24.7658418Z }, 2024-08-20T21:44:24.7658824Z { 2024-08-20T21:44:24.7659252Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7659824Z "size": 32, 2024-08-20T21:44:24.7660380Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7661014Z }, 2024-08-20T21:44:24.7661251Z { 2024-08-20T21:44:24.7661682Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7662242Z "size": 111, 2024-08-20T21:44:24.7662773Z "digest": "sha256:6d3ac49638ec41e1302ba5b1028a55629b0bb2cb2603212aba86d0058783ab50" 2024-08-20T21:44:24.7663404Z }, 2024-08-20T21:44:24.7663632Z { 2024-08-20T21:44:24.7664051Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7664598Z "size": 473, 2024-08-20T21:44:24.7665241Z "digest": "sha256:c5c5dbfb1dcc6e7ef2cabd2a583442e21c5b95a0cb625f0766592fff0b807d95" 2024-08-20T21:44:24.7665878Z }, 2024-08-20T21:44:24.7666079Z { 2024-08-20T21:44:24.7666490Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7667024Z "size": 32, 2024-08-20T21:44:24.7667549Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7668162Z }, 2024-08-20T21:44:24.7668364Z { 2024-08-20T21:44:24.7668776Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7669314Z "size": 112, 2024-08-20T21:44:24.7669836Z "digest": "sha256:327bd4a08d3c4e9ae265be2b8ce0d8ce4e4016f60e4c02a36c638789adf49ea5" 2024-08-20T21:44:24.7670456Z }, 2024-08-20T21:44:24.7670669Z { 2024-08-20T21:44:24.7671213Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7671726Z "size": 566, 2024-08-20T21:44:24.7672233Z "digest": "sha256:6c520cbfb1595425e6e07ad7e2952222365e91d02cf25a4ec1f8565279b941de" 2024-08-20T21:44:24.7672809Z }, 2024-08-20T21:44:24.7673002Z { 2024-08-20T21:44:24.7673397Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7673917Z "size": 43163722, 2024-08-20T21:44:24.7674446Z "digest": "sha256:a8cda4c989262a43930b0bf2507986800779f6a6ec65f8ef64bbfa72c5ca5bf6" 2024-08-20T21:44:24.7675009Z }, 2024-08-20T21:44:24.7675221Z { 2024-08-20T21:44:24.7675616Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7676129Z "size": 106, 2024-08-20T21:44:24.7676604Z "digest": "sha256:3d995c530b49c1eecd803a2cfde69821f88cc0b8b5429d8955a65319217337d6" 2024-08-20T21:44:24.7677171Z }, 2024-08-20T21:44:24.7677363Z { 2024-08-20T21:44:24.7677732Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7678258Z "size": 295, 2024-08-20T21:44:24.7678755Z "digest": "sha256:51635df36934a1409214ed1d1c03c997d8597887f34e547bca5fb5373e509ca8" 2024-08-20T21:44:24.7679327Z }, 2024-08-20T21:44:24.7679513Z { 2024-08-20T21:44:24.7680080Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7680627Z "size": 32, 2024-08-20T21:44:24.7681137Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7681728Z }, 2024-08-20T21:44:24.7681927Z { 2024-08-20T21:44:24.7682323Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7682932Z "size": 106, 2024-08-20T21:44:24.7683456Z "digest": "sha256:58f9d093cf6898c8d0fe441806e8afb6fcf8180d6ce0e61f3f8e1a5d27872b95" 2024-08-20T21:44:24.7684061Z }, 2024-08-20T21:44:24.7684272Z { 2024-08-20T21:44:24.7684664Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7685188Z "size": 425, 2024-08-20T21:44:24.7685709Z "digest": "sha256:db291b17fdf0ea8442b4ee4ccf970b6bcbbf230840366e5dfe62faf43175568d" 2024-08-20T21:44:24.7686321Z }, 2024-08-20T21:44:24.7686517Z { 2024-08-20T21:44:24.7686915Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7687440Z "size": 19308700, 2024-08-20T21:44:24.7687959Z "digest": "sha256:4390932d47dc99aad011fc417cb24b8cd895f84e0247e7b1930a974848f4ae49" 2024-08-20T21:44:24.7688537Z }, 2024-08-20T21:44:24.7688745Z { 2024-08-20T21:44:24.7689138Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7689657Z "size": 108, 2024-08-20T21:44:24.7690158Z "digest": "sha256:d959da1ed4a86d0902fc218b87883ea901725726db56d746621bfeb206d18ad6" 2024-08-20T21:44:24.7690746Z }, 2024-08-20T21:44:24.7690943Z { 2024-08-20T21:44:24.7691330Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7691852Z "size": 438, 2024-08-20T21:44:24.7692346Z "digest": "sha256:82769435ccd5317b20def22a2e4295f283bac4ed179599487713032c3609f189" 2024-08-20T21:44:24.7692929Z }, 2024-08-20T21:44:24.7693120Z { 2024-08-20T21:44:24.7693684Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7694182Z "size": 701, 2024-08-20T21:44:24.7694690Z "digest": "sha256:2cebdfe4c5a8da099cc35bcc957374e8545fed39fbf41715974c4d4c1ec3efa9" 2024-08-20T21:44:24.7695273Z }, 2024-08-20T21:44:24.7695489Z { 2024-08-20T21:44:24.7695875Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7696385Z "size": 141, 2024-08-20T21:44:24.7696882Z "digest": "sha256:50078e7792d3b05b7c50ff90f27eb679dee945052c25d2fa8590cbd90dee9ac0" 2024-08-20T21:44:24.7697445Z }, 2024-08-20T21:44:24.7697634Z { 2024-08-20T21:44:24.7698003Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7698598Z "size": 135, 2024-08-20T21:44:24.7699273Z "digest": "sha256:557fee3a8871f019075f948770fb5f0860218079c7dd47fcc7982499991fb2a1" 2024-08-20T21:44:24.7699878Z }, 2024-08-20T21:44:24.7700080Z { 2024-08-20T21:44:24.7700498Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7701050Z "size": 4841634229, 2024-08-20T21:44:24.7701588Z "digest": "sha256:cc0b3fc4f447d5036748372379999ebaac2ba414c77302c2910751054210e216" 2024-08-20T21:44:24.7702177Z }, 2024-08-20T21:44:24.7702389Z { 2024-08-20T21:44:24.7702824Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7703365Z "size": 194, 2024-08-20T21:44:24.7703880Z "digest": "sha256:f18a0119669a20a0f6987e879f10f84bd03b89b7865214f4904cd1b9987d0b0e" 2024-08-20T21:44:24.7704493Z }, 2024-08-20T21:44:24.7704719Z { 2024-08-20T21:44:24.7705133Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7705693Z "size": 1379, 2024-08-20T21:44:24.7706239Z "digest": "sha256:6197a6a036aca1f62c286b2b81941a497fa5ffed336bca2675084fe69f48f405" 2024-08-20T21:44:24.7706876Z }, 2024-08-20T21:44:24.7707089Z { 2024-08-20T21:44:24.7707510Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7708070Z "size": 701, 2024-08-20T21:44:24.7708621Z "digest": "sha256:2cebdfe4c5a8da099cc35bcc957374e8545fed39fbf41715974c4d4c1ec3efa9" 2024-08-20T21:44:24.7709272Z }, 2024-08-20T21:44:24.7709506Z { 2024-08-20T21:44:24.7709936Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7710483Z "size": 139, 2024-08-20T21:44:24.7711148Z "digest": "sha256:ebdeb237156e5f92ea7a5aa15d85a86b5c64ae0e6ca749609008f2e6eee128b8" 2024-08-20T21:44:24.7711816Z }, 2024-08-20T21:44:24.7712038Z { 2024-08-20T21:44:24.7712424Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7712954Z "size": 3733332465, 2024-08-20T21:44:24.7713497Z "digest": "sha256:fd5d0d6ba583ac4b9de591cc4925403acd2588d10fc04114b681beb022597580" 2024-08-20T21:44:24.7714097Z }, 2024-08-20T21:44:24.7714301Z { 2024-08-20T21:44:24.7714707Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7715231Z "size": 157, 2024-08-20T21:44:24.7715722Z "digest": "sha256:6b068755e588d5e5994af07fe71bd40adbbc544f6238dea5511e69862a5236dd" 2024-08-20T21:44:24.7716310Z }, 2024-08-20T21:44:24.7716531Z { 2024-08-20T21:44:24.7716934Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7717443Z "size": 849, 2024-08-20T21:44:24.7717940Z "digest": "sha256:371497d14c4e377a762eec74e0c7575051758f8aa74cdc2f2436fb382df849d3" 2024-08-20T21:44:24.7718524Z }, 2024-08-20T21:44:24.7718741Z { 2024-08-20T21:44:24.7719129Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7719652Z "size": 701, 2024-08-20T21:44:24.7720174Z "digest": "sha256:2cebdfe4c5a8da099cc35bcc957374e8545fed39fbf41715974c4d4c1ec3efa9" 2024-08-20T21:44:24.7720779Z }, 2024-08-20T21:44:24.7720978Z { 2024-08-20T21:44:24.7721383Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7721908Z "size": 139, 2024-08-20T21:44:24.7722386Z "digest": "sha256:4b50cb25a6ac9133e2c7894845496771a2f9b23eb35a58a778bb989f9bfbac5e" 2024-08-20T21:44:24.7723053Z }, 2024-08-20T21:44:24.7723278Z { 2024-08-20T21:44:24.7723686Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7724193Z "size": 32, 2024-08-20T21:44:24.7724707Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7725305Z }, 2024-08-20T21:44:24.7725533Z { 2024-08-20T21:44:24.7725915Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7726445Z "size": 159, 2024-08-20T21:44:24.7726956Z "digest": "sha256:889d8139c009597aca470a64bf7aaea6d509fd8ee9a96187fa784a1d9931f350" 2024-08-20T21:44:24.7727529Z }, 2024-08-20T21:44:24.7727745Z { 2024-08-20T21:44:24.7728149Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7728681Z "size": 907, 2024-08-20T21:44:24.7729187Z "digest": "sha256:2bcabd118f67bf8b2b8c6eb97e245fef2c7753e07e4ecf062b3cf6c905bdea81" 2024-08-20T21:44:24.7729797Z }, 2024-08-20T21:44:24.7730019Z { 2024-08-20T21:44:24.7730417Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7730922Z "size": 701, 2024-08-20T21:44:24.7731449Z "digest": "sha256:2cebdfe4c5a8da099cc35bcc957374e8545fed39fbf41715974c4d4c1ec3efa9" 2024-08-20T21:44:24.7732052Z }, 2024-08-20T21:44:24.7732254Z { 2024-08-20T21:44:24.7732651Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7733196Z "size": 134, 2024-08-20T21:44:24.7733715Z "digest": "sha256:1d831c1ea7c74b810f149c31d6eebc2d2bed6a5fb9179a9b230fec1c85f1094e" 2024-08-20T21:44:24.7734301Z }, 2024-08-20T21:44:24.7734519Z { 2024-08-20T21:44:24.7734928Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7735454Z "size": 32, 2024-08-20T21:44:24.7735946Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7736538Z }, 2024-08-20T21:44:24.7736758Z { 2024-08-20T21:44:24.7737167Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7737679Z "size": 158, 2024-08-20T21:44:24.7738187Z "digest": "sha256:d29d428148005386a1f6f19dd2590d9cbf58a6c1f94898e1e6ac92ea666f1641" 2024-08-20T21:44:24.7739082Z }, 2024-08-20T21:44:24.7739302Z { 2024-08-20T21:44:24.7739739Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7740311Z "size": 1579, 2024-08-20T21:44:24.7740947Z "digest": "sha256:10df3ed01753e29d9ea7d799510d1d4534671a9a4ac53db74998c1ba9c38a48c" 2024-08-20T21:44:24.7741562Z }, 2024-08-20T21:44:24.7741802Z { 2024-08-20T21:44:24.7742238Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7742809Z "size": 32, 2024-08-20T21:44:24.7743516Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7744167Z }, 2024-08-20T21:44:24.7744404Z { 2024-08-20T21:44:24.7744823Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7745389Z "size": 136, 2024-08-20T21:44:24.7746113Z "digest": "sha256:3b3541d6658c782e6cf71ee8b9268b351f32109e8571447dc8993e649742d905" 2024-08-20T21:44:24.7746734Z }, 2024-08-20T21:44:24.7746956Z { 2024-08-20T21:44:24.7747385Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7747948Z "size": 378, 2024-08-20T21:44:24.7748485Z "digest": "sha256:5e997601a6320f613b759c5684ae83fa39646d3c85a349797faba2e4fbc70d83" 2024-08-20T21:44:24.7749093Z }, 2024-08-20T21:44:24.7749325Z { 2024-08-20T21:44:24.7749754Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7750312Z "size": 32, 2024-08-20T21:44:24.7750842Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7751481Z }, 2024-08-20T21:44:24.7751717Z { 2024-08-20T21:44:24.7752130Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7752796Z "size": 104, 2024-08-20T21:44:24.7753338Z "digest": "sha256:d2924613594f110f7aa5b1681ba6bc1044cd355b5eeb17944cd11531d33a1552" 2024-08-20T21:44:24.7753966Z }, 2024-08-20T21:44:24.7754179Z { 2024-08-20T21:44:24.7754609Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7755174Z "size": 1841, 2024-08-20T21:44:24.7755815Z "digest": "sha256:f66c8294ffd00da25739af287f89a292ee2b06597012e6abac6947e2197961ae" 2024-08-20T21:44:24.7756392Z }, 2024-08-20T21:44:24.7756612Z { 2024-08-20T21:44:24.7757016Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7757524Z "size": 7529594, 2024-08-20T21:44:24.7758047Z "digest": "sha256:9da0ec93b0d7d37059844ca82dd8b0ce33bdd51614cc9bd66c391ffe64094ac7" 2024-08-20T21:44:24.7758644Z }, 2024-08-20T21:44:24.7758861Z { 2024-08-20T21:44:24.7759243Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7759765Z "size": 106, 2024-08-20T21:44:24.7760263Z "digest": "sha256:d476e4b8627568a99c85e93d60a7ce5676a42e9ee10a5b1054348658d77f56f3" 2024-08-20T21:44:24.7760842Z }, 2024-08-20T21:44:24.7761046Z { 2024-08-20T21:44:24.7761450Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7761982Z "size": 163, 2024-08-20T21:44:24.7762483Z "digest": "sha256:397c513e7e07f5c83f0d0c8c407a7bdc3e7167a57744262cd5a1282ee451d52d" 2024-08-20T21:44:24.7763051Z }, 2024-08-20T21:44:24.7763272Z { 2024-08-20T21:44:24.7763676Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7764184Z "size": 7943, 2024-08-20T21:44:24.7764695Z "digest": "sha256:5d6d58e6432b6e8048d3f6cccb787a1cfe9d1203f742a9f0fddfb00338cccae7" 2024-08-20T21:44:24.7765295Z }, 2024-08-20T21:44:24.7765513Z { 2024-08-20T21:44:24.7765894Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7766414Z "size": 8066, 2024-08-20T21:44:24.7766944Z "digest": "sha256:f9db7741af1464b9e28a0bab667a6b4559ab7f3d319add7a50bc1dcc2b1e97fe" 2024-08-20T21:44:24.7767552Z }, 2024-08-20T21:44:24.7767751Z { 2024-08-20T21:44:24.7768154Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7768680Z "size": 303, 2024-08-20T21:44:24.7769163Z "digest": "sha256:c0f5c95392e3593f2bccab197736341bfef11427975492f581c088b8e21401a1" 2024-08-20T21:44:24.7769740Z }, 2024-08-20T21:44:24.7770044Z { 2024-08-20T21:44:24.7770453Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7770958Z "size": 32, 2024-08-20T21:44:24.7771464Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7772061Z }, 2024-08-20T21:44:24.7772277Z { 2024-08-20T21:44:24.7772659Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7773181Z "size": 108, 2024-08-20T21:44:24.7773684Z "digest": "sha256:d6d4b368f5e67517827a02d0e324ced915843fa69780cbf87cac205ffcf04032" 2024-08-20T21:44:24.7774267Z }, 2024-08-20T21:44:24.7774465Z { 2024-08-20T21:44:24.7774868Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7775396Z "size": 54145664, 2024-08-20T21:44:24.7775906Z "digest": "sha256:6e341d8be4cd61d3439f2becbc390d9d27179704631d29225ba4c01bc518ff56" 2024-08-20T21:44:24.7776496Z }, 2024-08-20T21:44:24.7776714Z { 2024-08-20T21:44:24.7777121Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:44:24.7777634Z "size": 32, 2024-08-20T21:44:24.7778145Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:44:24.7779006Z } 2024-08-20T21:44:24.7779243Z ] 2024-08-20T21:44:24.7779513Z } 2024-08-20T21:44:24.7896678Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T21:44:24.7897148Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T21:44:24.7897683Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T21:44:24.7904486Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:24.7904974Z env: 2024-08-20T21:44:24.7905247Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:24.7906383Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.7907568Z ##[endgroup] 2024-08-20T21:44:24.7933534Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.7984060Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T21:44:24.7984649Z with: 2024-08-20T21:44:24.7985693Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.7987015Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:24.7987541Z env: 2024-08-20T21:44:24.7987794Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:24.7988132Z ##[endgroup] 2024-08-20T21:44:24.8003850Z ##[group]Run set -x 2024-08-20T21:44:24.8004170Z set -x 2024-08-20T21:44:24.8004455Z set +e 2024-08-20T21:44:24.8004722Z  2024-08-20T21:44:24.8004977Z login() { 2024-08-20T21:44:24.8005612Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T21:44:24.8006335Z } 2024-08-20T21:44:24.8006585Z  2024-08-20T21:44:24.8006873Z retry () { 2024-08-20T21:44:24.8007315Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T21:44:24.8007730Z } 2024-08-20T21:44:24.8007985Z  2024-08-20T21:44:24.8008280Z retry login "${DOCKER_REGISTRY}" 2024-08-20T21:44:24.8008665Z  2024-08-20T21:44:24.8008917Z set -e 2024-08-20T21:44:24.8009374Z # ignore output since only exit code is used for conditional 2024-08-20T21:44:24.8010073Z # only pull docker image if it's not available locally 2024-08-20T21:44:24.8010827Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T21:44:24.8011523Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T21:44:24.8011955Z fi 2024-08-20T21:44:24.8017536Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:44:24.8017988Z env: 2024-08-20T21:44:24.8018658Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:44:24.8019962Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:24.8021264Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:24.8021786Z ##[endgroup] 2024-08-20T21:44:24.8045887Z + set +e 2024-08-20T21:44:24.8046491Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:24.8047144Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:24.8048851Z + aws ecr get-login-password --region us-east-1 2024-08-20T21:44:24.8050366Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:44:25.3050678Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:44:25.3051517Z Configure a credential helper to remove this warning. See 2024-08-20T21:44:25.3052363Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:44:25.3052936Z 2024-08-20T21:44:25.3054129Z Login Succeeded 2024-08-20T21:44:25.3065684Z + set -e 2024-08-20T21:44:25.3067019Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:25.3197158Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:25.3199306Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:44:25.5510192Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks 2024-08-20T21:44:25.5511158Z 762bedf4b1b7: Pulling fs layer 2024-08-20T21:44:25.5511556Z 347beb78d39b: Pulling fs layer 2024-08-20T21:44:25.5512073Z 71b614615bcb: Pulling fs layer 2024-08-20T21:44:25.5512779Z f96c8558662d: Pulling fs layer 2024-08-20T21:44:25.5513189Z c81ff9d20d36: Pulling fs layer 2024-08-20T21:44:25.5513575Z 151e9a9d2894: Pulling fs layer 2024-08-20T21:44:25.5513952Z 8538dcd89f0c: Pulling fs layer 2024-08-20T21:44:25.5514329Z cd13007f6f09: Pulling fs layer 2024-08-20T21:44:25.5514867Z 74fd96b6ebd9: Pulling fs layer 2024-08-20T21:44:25.5515290Z ce4e1800db98: Pulling fs layer 2024-08-20T21:44:25.5515675Z 402b067f30bf: Pulling fs layer 2024-08-20T21:44:25.5516055Z 2cebdfe4c5a8: Pulling fs layer 2024-08-20T21:44:25.5516421Z c6e13ff8aa9a: Pulling fs layer 2024-08-20T21:44:25.5516786Z f96c8558662d: Waiting 2024-08-20T21:44:25.5517202Z 4f4fb700ef54: Pulling fs layer 2024-08-20T21:44:25.5517792Z e238df05ddcc: Pulling fs layer 2024-08-20T21:44:25.5518213Z c81ff9d20d36: Waiting 2024-08-20T21:44:25.5518645Z a88675bdc723: Pulling fs layer 2024-08-20T21:44:25.5519292Z f4cb80c411d1: Pulling fs layer 2024-08-20T21:44:25.5519671Z 151e9a9d2894: Waiting 2024-08-20T21:44:25.5541828Z c9589ebd9ae7: Pulling fs layer 2024-08-20T21:44:25.5542283Z dd51c681db14: Pulling fs layer 2024-08-20T21:44:25.5542682Z fc983e33a069: Pulling fs layer 2024-08-20T21:44:25.5543049Z bc705d688c77: Pulling fs layer 2024-08-20T21:44:25.5543723Z 8538dcd89f0c: Waiting 2024-08-20T21:44:25.5544057Z 402b067f30bf: Waiting 2024-08-20T21:44:25.5544401Z 99f6c1ac7582: Pulling fs layer 2024-08-20T21:44:25.5544769Z c409d27a5a95: Pulling fs layer 2024-08-20T21:44:25.5545149Z a3a398ca1120: Pulling fs layer 2024-08-20T21:44:25.5545527Z 314875f5fae4: Pulling fs layer 2024-08-20T21:44:25.5545875Z cd13007f6f09: Waiting 2024-08-20T21:44:25.5546200Z 2cebdfe4c5a8: Waiting 2024-08-20T21:44:25.5546534Z 8a5e90dafefa: Pulling fs layer 2024-08-20T21:44:25.5546919Z e3b946ab3378: Pulling fs layer 2024-08-20T21:44:25.5547277Z 14a85a990f71: Pulling fs layer 2024-08-20T21:44:25.5547645Z 16d809b167d4: Pulling fs layer 2024-08-20T21:44:25.5548083Z 9b1a95eedb83: Pulling fs layer 2024-08-20T21:44:25.5548701Z b35ca0542723: Pulling fs layer 2024-08-20T21:44:25.5549068Z 74fd96b6ebd9: Waiting 2024-08-20T21:44:25.5549392Z bad10ac3a627: Pulling fs layer 2024-08-20T21:44:25.5549776Z 943da2164c7a: Pulling fs layer 2024-08-20T21:44:25.5550142Z c6e13ff8aa9a: Waiting 2024-08-20T21:44:25.5550458Z 2d276441bd8a: Pulling fs layer 2024-08-20T21:44:25.5550825Z 4f4fb700ef54: Waiting 2024-08-20T21:44:25.5551138Z e3b946ab3378: Waiting 2024-08-20T21:44:25.5551451Z 14a85a990f71: Waiting 2024-08-20T21:44:25.5551743Z 16d809b167d4: Waiting 2024-08-20T21:44:25.5552066Z a3ef8d7d02be: Pulling fs layer 2024-08-20T21:44:25.5552429Z dd51c681db14: Waiting 2024-08-20T21:44:25.5552744Z 6aaed7a78749: Pulling fs layer 2024-08-20T21:44:25.5553113Z e238df05ddcc: Waiting 2024-08-20T21:44:25.5553443Z 944e78083916: Pulling fs layer 2024-08-20T21:44:25.5553810Z 9602de36240c: Pulling fs layer 2024-08-20T21:44:25.5554152Z 99f6c1ac7582: Waiting 2024-08-20T21:44:25.5554484Z 6d3ac49638ec: Pulling fs layer 2024-08-20T21:44:25.5554863Z c5c5dbfb1dcc: Pulling fs layer 2024-08-20T21:44:25.5555236Z fc983e33a069: Waiting 2024-08-20T21:44:25.5555547Z 9b1a95eedb83: Waiting 2024-08-20T21:44:25.5555857Z a88675bdc723: Waiting 2024-08-20T21:44:25.5556166Z c409d27a5a95: Waiting 2024-08-20T21:44:25.5556474Z 327bd4a08d3c: Pulling fs layer 2024-08-20T21:44:25.5556852Z 6c520cbfb159: Pulling fs layer 2024-08-20T21:44:25.5557229Z a8cda4c98926: Pulling fs layer 2024-08-20T21:44:25.5557588Z 314875f5fae4: Waiting 2024-08-20T21:44:25.5557899Z 3d995c530b49: Pulling fs layer 2024-08-20T21:44:25.5558259Z 8a5e90dafefa: Waiting 2024-08-20T21:44:25.5558586Z 51635df36934: Pulling fs layer 2024-08-20T21:44:25.5558950Z c9589ebd9ae7: Waiting 2024-08-20T21:44:25.5559250Z b35ca0542723: Waiting 2024-08-20T21:44:25.5559564Z bad10ac3a627: Waiting 2024-08-20T21:44:25.5559892Z 58f9d093cf68: Pulling fs layer 2024-08-20T21:44:25.5560261Z db291b17fdf0: Pulling fs layer 2024-08-20T21:44:25.5560630Z 6aaed7a78749: Waiting 2024-08-20T21:44:25.5560959Z 4390932d47dc: Pulling fs layer 2024-08-20T21:44:25.5561341Z d959da1ed4a8: Pulling fs layer 2024-08-20T21:44:25.5561839Z 3d995c530b49: Waiting 2024-08-20T21:44:25.5562160Z 944e78083916: Waiting 2024-08-20T21:44:25.5562493Z 82769435ccd5: Pulling fs layer 2024-08-20T21:44:25.5562840Z 6d3ac49638ec: Waiting 2024-08-20T21:44:25.5563159Z 9602de36240c: Waiting 2024-08-20T21:44:25.5563473Z f4cb80c411d1: Waiting 2024-08-20T21:44:25.5563795Z a3a398ca1120: Waiting 2024-08-20T21:44:25.5564096Z 6c520cbfb159: Waiting 2024-08-20T21:44:25.5564424Z 50078e7792d3: Pulling fs layer 2024-08-20T21:44:25.5564784Z 327bd4a08d3c: Waiting 2024-08-20T21:44:25.5565100Z a3ef8d7d02be: Waiting 2024-08-20T21:44:25.5565413Z 557fee3a8871: Pulling fs layer 2024-08-20T21:44:25.5565791Z cc0b3fc4f447: Pulling fs layer 2024-08-20T21:44:25.5566167Z f18a0119669a: Pulling fs layer 2024-08-20T21:44:25.5566522Z 6197a6a036ac: Pulling fs layer 2024-08-20T21:44:25.5566877Z 51635df36934: Waiting 2024-08-20T21:44:25.5567208Z ebdeb237156e: Pulling fs layer 2024-08-20T21:44:25.5567590Z fd5d0d6ba583: Pulling fs layer 2024-08-20T21:44:25.5567939Z a8cda4c98926: Waiting 2024-08-20T21:44:25.5568257Z 50078e7792d3: Waiting 2024-08-20T21:44:25.5568589Z 6b068755e588: Pulling fs layer 2024-08-20T21:44:25.5568950Z 6197a6a036ac: Waiting 2024-08-20T21:44:25.5569247Z db291b17fdf0: Waiting 2024-08-20T21:44:25.5569572Z 371497d14c4e: Pulling fs layer 2024-08-20T21:44:25.5569956Z 4b50cb25a6ac: Pulling fs layer 2024-08-20T21:44:25.5570297Z 371497d14c4e: Waiting 2024-08-20T21:44:25.5570607Z fd5d0d6ba583: Waiting 2024-08-20T21:44:25.5570934Z 889d8139c009: Pulling fs layer 2024-08-20T21:44:25.5571293Z d959da1ed4a8: Waiting 2024-08-20T21:44:25.5571603Z 2bcabd118f67: Pulling fs layer 2024-08-20T21:44:25.5571968Z 2d276441bd8a: Waiting 2024-08-20T21:44:25.5572296Z 1d831c1ea7c7: Pulling fs layer 2024-08-20T21:44:25.5572635Z 889d8139c009: Waiting 2024-08-20T21:44:25.5572948Z 82769435ccd5: Waiting 2024-08-20T21:44:25.5573260Z 1d831c1ea7c7: Waiting 2024-08-20T21:44:25.5573585Z d29d42814800: Pulling fs layer 2024-08-20T21:44:25.5574051Z 10df3ed01753: Pulling fs layer 2024-08-20T21:44:25.5574440Z 3b3541d6658c: Pulling fs layer 2024-08-20T21:44:25.5574811Z cc0b3fc4f447: Waiting 2024-08-20T21:44:25.5575132Z bc705d688c77: Waiting 2024-08-20T21:44:25.5575448Z 5e997601a632: Pulling fs layer 2024-08-20T21:44:25.5575811Z 10df3ed01753: Waiting 2024-08-20T21:44:25.5576130Z 3b3541d6658c: Waiting 2024-08-20T21:44:25.5576440Z d2924613594f: Pulling fs layer 2024-08-20T21:44:25.5576820Z f66c8294ffd0: Pulling fs layer 2024-08-20T21:44:25.5577200Z 9da0ec93b0d7: Pulling fs layer 2024-08-20T21:44:25.5577583Z d476e4b86275: Pulling fs layer 2024-08-20T21:44:25.5577942Z 397c513e7e07: Pulling fs layer 2024-08-20T21:44:25.5578450Z 557fee3a8871: Waiting 2024-08-20T21:44:25.5578796Z 5d6d58e6432b: Pulling fs layer 2024-08-20T21:44:25.5579160Z d2924613594f: Waiting 2024-08-20T21:44:25.5579454Z 6b068755e588: Waiting 2024-08-20T21:44:25.5579771Z d476e4b86275: Waiting 2024-08-20T21:44:25.5580091Z ebdeb237156e: Waiting 2024-08-20T21:44:25.5580411Z f9db7741af14: Pulling fs layer 2024-08-20T21:44:25.5580798Z c0f5c95392e3: Pulling fs layer 2024-08-20T21:44:25.5581168Z f9db7741af14: Waiting 2024-08-20T21:44:25.5581485Z c0f5c95392e3: Waiting 2024-08-20T21:44:25.5581783Z 5d6d58e6432b: Waiting 2024-08-20T21:44:25.5582115Z d6d4b368f5e6: Pulling fs layer 2024-08-20T21:44:25.5582502Z 6e341d8be4cd: Pulling fs layer 2024-08-20T21:44:25.5582852Z 4b50cb25a6ac: Waiting 2024-08-20T21:44:25.5583171Z f18a0119669a: Waiting 2024-08-20T21:44:25.5583481Z d29d42814800: Waiting 2024-08-20T21:44:25.5583791Z 5e997601a632: Waiting 2024-08-20T21:44:25.5584083Z 397c513e7e07: Waiting 2024-08-20T21:44:25.5584395Z d6d4b368f5e6: Waiting 2024-08-20T21:44:25.6254593Z 347beb78d39b: Verifying Checksum 2024-08-20T21:44:25.6255165Z 347beb78d39b: Download complete 2024-08-20T21:44:25.6940933Z f96c8558662d: Verifying Checksum 2024-08-20T21:44:25.6941394Z f96c8558662d: Download complete 2024-08-20T21:44:25.7646408Z c81ff9d20d36: Download complete 2024-08-20T21:44:25.8368662Z 151e9a9d2894: Download complete 2024-08-20T21:44:25.9274653Z 8538dcd89f0c: Download complete 2024-08-20T21:44:25.9900682Z 762bedf4b1b7: Verifying Checksum 2024-08-20T21:44:25.9901145Z 762bedf4b1b7: Download complete 2024-08-20T21:44:26.0086734Z cd13007f6f09: Verifying Checksum 2024-08-20T21:44:26.0087220Z cd13007f6f09: Download complete 2024-08-20T21:44:26.0686696Z ce4e1800db98: Download complete 2024-08-20T21:44:26.1364128Z 402b067f30bf: Download complete 2024-08-20T21:44:26.2094579Z 2cebdfe4c5a8: Verifying Checksum 2024-08-20T21:44:26.2095159Z 2cebdfe4c5a8: Download complete 2024-08-20T21:44:26.7078385Z 762bedf4b1b7: Pull complete 2024-08-20T21:44:26.7247568Z 347beb78d39b: Pull complete 2024-08-20T21:44:27.2575819Z 74fd96b6ebd9: Verifying Checksum 2024-08-20T21:44:27.2576270Z 74fd96b6ebd9: Download complete 2024-08-20T21:44:27.2689575Z 4f4fb700ef54: Verifying Checksum 2024-08-20T21:44:27.2690005Z 4f4fb700ef54: Download complete 2024-08-20T21:44:27.3438622Z e238df05ddcc: Verifying Checksum 2024-08-20T21:44:27.3439060Z e238df05ddcc: Download complete 2024-08-20T21:44:27.4463911Z a88675bdc723: Download complete 2024-08-20T21:44:27.5294603Z f4cb80c411d1: Verifying Checksum 2024-08-20T21:44:27.5295065Z f4cb80c411d1: Download complete 2024-08-20T21:44:27.7155263Z c9589ebd9ae7: Verifying Checksum 2024-08-20T21:44:27.7155752Z c9589ebd9ae7: Download complete 2024-08-20T21:44:27.7910029Z dd51c681db14: Download complete 2024-08-20T21:44:27.8546808Z fc983e33a069: Verifying Checksum 2024-08-20T21:44:27.8547265Z fc983e33a069: Download complete 2024-08-20T21:44:27.9211565Z bc705d688c77: Verifying Checksum 2024-08-20T21:44:27.9212148Z bc705d688c77: Download complete 2024-08-20T21:44:27.9880814Z 99f6c1ac7582: Download complete 2024-08-20T21:44:28.0582416Z c409d27a5a95: Verifying Checksum 2024-08-20T21:44:28.0582849Z c409d27a5a95: Download complete 2024-08-20T21:44:28.7308016Z 71b614615bcb: Verifying Checksum 2024-08-20T21:44:28.7308479Z 71b614615bcb: Download complete 2024-08-20T21:44:28.8168514Z 314875f5fae4: Verifying Checksum 2024-08-20T21:44:28.8169096Z 314875f5fae4: Download complete 2024-08-20T21:44:28.8947588Z 8a5e90dafefa: Verifying Checksum 2024-08-20T21:44:28.8948080Z 8a5e90dafefa: Download complete 2024-08-20T21:44:28.9722370Z e3b946ab3378: Verifying Checksum 2024-08-20T21:44:28.9722889Z e3b946ab3378: Download complete 2024-08-20T21:44:29.0427252Z 14a85a990f71: Verifying Checksum 2024-08-20T21:44:29.0427716Z 14a85a990f71: Download complete 2024-08-20T21:44:29.1169871Z 16d809b167d4: Verifying Checksum 2024-08-20T21:44:29.1170356Z 16d809b167d4: Download complete 2024-08-20T21:44:29.1222973Z a3a398ca1120: Verifying Checksum 2024-08-20T21:44:29.1223399Z a3a398ca1120: Download complete 2024-08-20T21:44:29.1899731Z b35ca0542723: Download complete 2024-08-20T21:44:29.2616872Z bad10ac3a627: Verifying Checksum 2024-08-20T21:44:29.2617368Z bad10ac3a627: Download complete 2024-08-20T21:44:29.3358525Z 943da2164c7a: Verifying Checksum 2024-08-20T21:44:29.3359003Z 943da2164c7a: Download complete 2024-08-20T21:44:29.4098667Z 2d276441bd8a: Download complete 2024-08-20T21:44:29.5132684Z a3ef8d7d02be: Verifying Checksum 2024-08-20T21:44:29.5133190Z a3ef8d7d02be: Download complete 2024-08-20T21:44:29.5758414Z 6aaed7a78749: Verifying Checksum 2024-08-20T21:44:29.5758892Z 6aaed7a78749: Download complete 2024-08-20T21:44:29.6515817Z 944e78083916: Verifying Checksum 2024-08-20T21:44:29.6516284Z 944e78083916: Download complete 2024-08-20T21:44:29.7189095Z 9602de36240c: Download complete 2024-08-20T21:44:29.7855001Z 6d3ac49638ec: Verifying Checksum 2024-08-20T21:44:29.7855475Z 6d3ac49638ec: Download complete 2024-08-20T21:44:29.8563858Z c5c5dbfb1dcc: Download complete 2024-08-20T21:44:29.9331410Z 327bd4a08d3c: Verifying Checksum 2024-08-20T21:44:29.9331946Z 327bd4a08d3c: Download complete 2024-08-20T21:44:30.0150722Z 6c520cbfb159: Verifying Checksum 2024-08-20T21:44:30.0151432Z 6c520cbfb159: Download complete 2024-08-20T21:44:30.5025499Z a8cda4c98926: Verifying Checksum 2024-08-20T21:44:30.5026027Z a8cda4c98926: Download complete 2024-08-20T21:44:30.5742977Z 3d995c530b49: Verifying Checksum 2024-08-20T21:44:30.5743640Z 3d995c530b49: Download complete 2024-08-20T21:44:30.6758859Z 51635df36934: Verifying Checksum 2024-08-20T21:44:30.6759343Z 51635df36934: Download complete 2024-08-20T21:44:30.7441432Z 58f9d093cf68: Verifying Checksum 2024-08-20T21:44:30.7441877Z 58f9d093cf68: Download complete 2024-08-20T21:44:30.8131404Z db291b17fdf0: Download complete 2024-08-20T21:44:31.0581169Z 4390932d47dc: Verifying Checksum 2024-08-20T21:44:31.0581876Z 4390932d47dc: Download complete 2024-08-20T21:44:31.1188804Z d959da1ed4a8: Download complete 2024-08-20T21:44:31.1921437Z 82769435ccd5: Download complete 2024-08-20T21:44:31.2998147Z 50078e7792d3: Download complete 2024-08-20T21:44:31.3798261Z 557fee3a8871: Verifying Checksum 2024-08-20T21:44:31.3798756Z 557fee3a8871: Download complete 2024-08-20T21:44:35.2845707Z 9b1a95eedb83: Verifying Checksum 2024-08-20T21:44:35.2846947Z 9b1a95eedb83: Download complete 2024-08-20T21:44:35.2981971Z 71b614615bcb: Pull complete 2024-08-20T21:44:35.3594377Z f18a0119669a: Verifying Checksum 2024-08-20T21:44:35.3594837Z f18a0119669a: Download complete 2024-08-20T21:44:35.4289565Z 6197a6a036ac: Verifying Checksum 2024-08-20T21:44:35.4290757Z 6197a6a036ac: Download complete 2024-08-20T21:44:35.5359144Z ebdeb237156e: Verifying Checksum 2024-08-20T21:44:35.5359615Z ebdeb237156e: Download complete 2024-08-20T21:44:35.7396181Z f96c8558662d: Pull complete 2024-08-20T21:44:36.1686127Z c81ff9d20d36: Pull complete 2024-08-20T21:44:36.6302671Z 151e9a9d2894: Pull complete 2024-08-20T21:44:37.0894747Z 8538dcd89f0c: Pull complete 2024-08-20T21:44:37.5152701Z cd13007f6f09: Pull complete 2024-08-20T21:44:39.9405067Z 74fd96b6ebd9: Pull complete 2024-08-20T21:44:40.3776832Z ce4e1800db98: Pull complete 2024-08-20T21:44:40.8267248Z 402b067f30bf: Pull complete 2024-08-20T21:44:41.2905481Z 2cebdfe4c5a8: Pull complete 2024-08-20T21:44:52.8905039Z c6e13ff8aa9a: Verifying Checksum 2024-08-20T21:44:52.8905530Z c6e13ff8aa9a: Download complete 2024-08-20T21:44:52.9554619Z 6b068755e588: Verifying Checksum 2024-08-20T21:44:52.9555378Z 6b068755e588: Download complete 2024-08-20T21:44:53.0260578Z 371497d14c4e: Verifying Checksum 2024-08-20T21:44:53.0261021Z 371497d14c4e: Download complete 2024-08-20T21:44:53.1034044Z 4b50cb25a6ac: Verifying Checksum 2024-08-20T21:44:53.1034526Z 4b50cb25a6ac: Download complete 2024-08-20T21:44:53.1741089Z 889d8139c009: Verifying Checksum 2024-08-20T21:44:53.1741570Z 889d8139c009: Download complete 2024-08-20T21:44:53.2531165Z 2bcabd118f67: Download complete 2024-08-20T21:44:53.3396932Z 1d831c1ea7c7: Verifying Checksum 2024-08-20T21:44:53.3397521Z 1d831c1ea7c7: Download complete 2024-08-20T21:44:53.4091765Z d29d42814800: Verifying Checksum 2024-08-20T21:44:53.4092212Z d29d42814800: Download complete 2024-08-20T21:44:53.4891343Z 10df3ed01753: Verifying Checksum 2024-08-20T21:44:53.4891825Z 10df3ed01753: Download complete 2024-08-20T21:44:53.5601443Z 3b3541d6658c: Verifying Checksum 2024-08-20T21:44:53.5601890Z 3b3541d6658c: Download complete 2024-08-20T21:44:53.6245457Z 5e997601a632: Verifying Checksum 2024-08-20T21:44:53.6246047Z 5e997601a632: Download complete 2024-08-20T21:44:53.7116477Z d2924613594f: Download complete 2024-08-20T21:44:53.7804884Z f66c8294ffd0: Verifying Checksum 2024-08-20T21:44:53.7805335Z f66c8294ffd0: Download complete 2024-08-20T21:44:53.9218768Z 9da0ec93b0d7: Verifying Checksum 2024-08-20T21:44:53.9219403Z 9da0ec93b0d7: Download complete 2024-08-20T21:44:54.0219255Z d476e4b86275: Download complete 2024-08-20T21:44:54.1049275Z 397c513e7e07: Verifying Checksum 2024-08-20T21:44:54.1049740Z 397c513e7e07: Download complete 2024-08-20T21:44:54.1648788Z 5d6d58e6432b: Verifying Checksum 2024-08-20T21:44:54.1649247Z 5d6d58e6432b: Download complete 2024-08-20T21:44:54.2359374Z f9db7741af14: Download complete 2024-08-20T21:44:54.3090382Z c0f5c95392e3: Verifying Checksum 2024-08-20T21:44:54.3090829Z c0f5c95392e3: Download complete 2024-08-20T21:44:54.3817786Z d6d4b368f5e6: Download complete 2024-08-20T21:44:54.9813922Z 6e341d8be4cd: Verifying Checksum 2024-08-20T21:44:54.9815204Z 6e341d8be4cd: Download complete 2024-08-20T21:45:12.9269863Z fd5d0d6ba583: Verifying Checksum 2024-08-20T21:45:12.9270678Z fd5d0d6ba583: Download complete 2024-08-20T21:45:19.8396162Z cc0b3fc4f447: Verifying Checksum 2024-08-20T21:45:19.8396677Z cc0b3fc4f447: Download complete 2024-08-20T21:45:30.8437828Z c6e13ff8aa9a: Pull complete 2024-08-20T21:45:31.3034145Z 4f4fb700ef54: Pull complete 2024-08-20T21:45:31.7722259Z e238df05ddcc: Pull complete 2024-08-20T21:45:32.2622753Z a88675bdc723: Pull complete 2024-08-20T21:45:32.6375109Z f4cb80c411d1: Pull complete 2024-08-20T21:45:33.1330027Z c9589ebd9ae7: Pull complete 2024-08-20T21:45:33.5578232Z dd51c681db14: Pull complete 2024-08-20T21:45:34.0162322Z fc983e33a069: Pull complete 2024-08-20T21:45:34.4454786Z bc705d688c77: Pull complete 2024-08-20T21:45:35.2016613Z 99f6c1ac7582: Pull complete 2024-08-20T21:45:35.6140518Z c409d27a5a95: Pull complete 2024-08-20T21:45:37.7539296Z a3a398ca1120: Pull complete 2024-08-20T21:45:38.1598338Z 314875f5fae4: Pull complete 2024-08-20T21:45:38.5944780Z 8a5e90dafefa: Pull complete 2024-08-20T21:45:39.0639751Z e3b946ab3378: Pull complete 2024-08-20T21:45:39.5289523Z 14a85a990f71: Pull complete 2024-08-20T21:45:39.9949060Z 16d809b167d4: Pull complete 2024-08-20T21:45:47.5853341Z 9b1a95eedb83: Pull complete 2024-08-20T21:45:47.8873793Z b35ca0542723: Pull complete 2024-08-20T21:45:48.2453276Z bad10ac3a627: Pull complete 2024-08-20T21:45:48.6253691Z 943da2164c7a: Pull complete 2024-08-20T21:45:48.9152650Z 2d276441bd8a: Pull complete 2024-08-20T21:45:49.3798066Z a3ef8d7d02be: Pull complete 2024-08-20T21:45:49.6862899Z 6aaed7a78749: Pull complete 2024-08-20T21:45:50.0149905Z 944e78083916: Pull complete 2024-08-20T21:45:50.1242143Z 9602de36240c: Pull complete 2024-08-20T21:45:50.7837784Z 6d3ac49638ec: Pull complete 2024-08-20T21:45:51.1231961Z c5c5dbfb1dcc: Pull complete 2024-08-20T21:45:51.4884116Z 327bd4a08d3c: Pull complete 2024-08-20T21:45:51.6828724Z 6c520cbfb159: Pull complete 2024-08-20T21:45:53.1576188Z a8cda4c98926: Pull complete 2024-08-20T21:45:53.3964293Z 3d995c530b49: Pull complete 2024-08-20T21:45:53.7300336Z 51635df36934: Pull complete 2024-08-20T21:45:54.4060600Z 58f9d093cf68: Pull complete 2024-08-20T21:45:54.8552108Z db291b17fdf0: Pull complete 2024-08-20T21:45:55.3763103Z 4390932d47dc: Pull complete 2024-08-20T21:45:55.8429503Z d959da1ed4a8: Pull complete 2024-08-20T21:45:56.1895935Z 82769435ccd5: Pull complete 2024-08-20T21:45:56.7216908Z 50078e7792d3: Pull complete 2024-08-20T21:45:56.8438633Z 557fee3a8871: Pull complete 2024-08-20T21:46:25.1395769Z cc0b3fc4f447: Pull complete 2024-08-20T21:46:25.5397391Z f18a0119669a: Pull complete 2024-08-20T21:46:25.9627471Z 6197a6a036ac: Pull complete 2024-08-20T21:46:26.7069814Z ebdeb237156e: Pull complete 2024-08-20T21:47:14.0157346Z fd5d0d6ba583: Pull complete 2024-08-20T21:47:14.4716522Z 6b068755e588: Pull complete 2024-08-20T21:47:14.9234327Z 371497d14c4e: Pull complete 2024-08-20T21:47:15.8097530Z 4b50cb25a6ac: Pull complete 2024-08-20T21:47:16.7050262Z 889d8139c009: Pull complete 2024-08-20T21:47:17.1485668Z 2bcabd118f67: Pull complete 2024-08-20T21:47:18.0140937Z 1d831c1ea7c7: Pull complete 2024-08-20T21:47:18.6942006Z d29d42814800: Pull complete 2024-08-20T21:47:19.1323514Z 10df3ed01753: Pull complete 2024-08-20T21:47:19.8947350Z 3b3541d6658c: Pull complete 2024-08-20T21:47:20.3254282Z 5e997601a632: Pull complete 2024-08-20T21:47:21.2256299Z d2924613594f: Pull complete 2024-08-20T21:47:21.6556401Z f66c8294ffd0: Pull complete 2024-08-20T21:47:22.1319345Z 9da0ec93b0d7: Pull complete 2024-08-20T21:47:22.5632344Z d476e4b86275: Pull complete 2024-08-20T21:47:22.7826021Z 397c513e7e07: Pull complete 2024-08-20T21:47:22.9819487Z 5d6d58e6432b: Pull complete 2024-08-20T21:47:23.4040446Z f9db7741af14: Pull complete 2024-08-20T21:47:23.8534617Z c0f5c95392e3: Pull complete 2024-08-20T21:47:24.6021168Z d6d4b368f5e6: Pull complete 2024-08-20T21:47:26.4779158Z 6e341d8be4cd: Pull complete 2024-08-20T21:47:27.0958230Z Digest: sha256:ed18f711327455e269271a17448384275684348a472ba289f00904bd14501fa9 2024-08-20T21:47:27.1931446Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:47:27.2468695Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:47:27.2528161Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T21:47:27.2529149Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T21:47:27.2536529Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:27.2537034Z env: 2024-08-20T21:47:27.2537314Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:27.2537664Z ##[endgroup] 2024-08-20T21:47:27.2787191Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T21:47:27.2787953Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T21:47:27.2788656Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T21:47:27.2789485Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:47:27.2795138Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:27.2795642Z env: 2024-08-20T21:47:27.2795912Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:27.2796251Z ##[endgroup] 2024-08-20T21:47:28.3677647Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T21:47:28.7596866Z Collecting psutil==5.9.1 2024-08-20T21:47:28.7958970Z Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB) 2024-08-20T21:47:28.9584776Z Collecting nvidia-ml-py==11.525.84 2024-08-20T21:47:28.9628636Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2024-08-20T21:47:29.1415922Z Installing collected packages: psutil, nvidia-ml-py 2024-08-20T21:47:29.3354146Z Successfully installed nvidia-ml-py-11.525.84 psutil-5.9.1 2024-08-20T21:47:29.5704017Z Prepare all required actions 2024-08-20T21:47:29.5704541Z Getting action download info 2024-08-20T21:47:29.6749757Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T21:47:29.9053834Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T21:47:30.0639479Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T21:47:30.0639954Z with: 2024-08-20T21:47:30.0640230Z name: linux-jammy-py3.8-gcc11-build 2024-08-20T21:47:30.0640633Z s3-bucket: gha-artifacts 2024-08-20T21:47:30.0640951Z env: 2024-08-20T21:47:30.0641198Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:30.0641502Z ##[endgroup] 2024-08-20T21:47:30.0721381Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T21:47:30.0721780Z with: 2024-08-20T21:47:30.0722046Z name: linux-jammy-py3.8-gcc11-build 2024-08-20T21:47:30.0722446Z s3-bucket: gha-artifacts 2024-08-20T21:47:30.0722794Z region: us-east-1 2024-08-20T21:47:30.0723045Z env: 2024-08-20T21:47:30.0723279Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:30.0723582Z ##[endgroup] 2024-08-20T21:47:30.5328084Z (node:43715) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T21:47:30.5329020Z 2024-08-20T21:47:30.5329276Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T21:47:30.5330065Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T21:47:30.5330997Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T21:47:30.6006697Z Found 1 objects with prefix pytorch/pytorch/10479310725/linux-jammy-py3.8-gcc11-build/ 2024-08-20T21:47:30.6007847Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T21:47:34.3389930Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T21:47:34.3396897Z Artifact download has finished successfully 2024-08-20T21:47:34.3648491Z ##[group]Run unzip -o artifacts.zip 2024-08-20T21:47:34.3648920Z unzip -o artifacts.zip 2024-08-20T21:47:34.3655258Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:34.3655724Z env: 2024-08-20T21:47:34.3655972Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:34.3656284Z ##[endgroup] 2024-08-20T21:47:34.4238704Z Archive: artifacts.zip 2024-08-20T21:47:34.4239087Z creating: dist/ 2024-08-20T21:47:35.3507293Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl 2024-08-20T21:47:35.3507979Z creating: build/custom_test_artifacts/ 2024-08-20T21:47:35.3508567Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T21:47:35.3509312Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T21:47:35.3510191Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:47:35.3511256Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:47:35.3512232Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T21:47:35.3513236Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:47:35.3514293Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:47:35.3515324Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:47:35.3516511Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:47:35.3517693Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:47:35.3518756Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:47:35.3519815Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:47:35.3521473Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:47:35.3522722Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:47:35.3523952Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:47:35.3525185Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:47:35.3526426Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:47:35.3527679Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:47:35.3528713Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:47:35.3529694Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T21:47:35.3530683Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T21:47:35.3531784Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T21:47:35.3533026Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T21:47:35.3534204Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T21:47:35.3535312Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T21:47:35.3536469Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T21:47:35.3537619Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T21:47:35.3538852Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T21:47:35.3540015Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T21:47:35.3541146Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T21:47:35.3547268Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T21:47:35.3676704Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T21:47:35.3677788Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T21:47:35.3678972Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T21:47:35.3680289Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T21:47:35.3681543Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T21:47:35.3682734Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T21:47:35.3683937Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T21:47:35.3685161Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T21:47:35.3686392Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T21:47:35.3687607Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T21:47:35.3688799Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T21:47:35.3699106Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T21:47:35.3774339Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T21:47:35.3775688Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:47:35.3776867Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:47:35.3777917Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T21:47:35.3778974Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T21:47:35.3779924Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T21:47:35.3780842Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T21:47:35.3781660Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T21:47:35.3782496Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T21:47:35.3889339Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T21:47:35.3947068Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T21:47:35.3947869Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T21:47:35.3948596Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T21:47:35.3949440Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:47:35.3950793Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:47:35.3951784Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T21:47:35.3952768Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:47:35.3953797Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:47:35.3954813Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:47:35.3955990Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:47:35.3957162Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:47:35.3958210Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:47:35.3959242Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:47:35.3960471Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:47:35.3961699Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:47:35.3962907Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:47:35.3964115Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:47:35.3965351Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:47:35.3966603Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:47:35.3967634Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:47:35.3968602Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T21:47:35.3969592Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T21:47:35.3970731Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T21:47:35.3972001Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T21:47:35.3973228Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T21:47:35.3974626Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T21:47:35.3975803Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T21:47:35.3977004Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T21:47:35.3978289Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T21:47:35.3979483Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T21:47:35.3980643Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T21:47:35.3986909Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T21:47:35.4046506Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T21:47:35.4047817Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:47:35.4048975Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:47:35.4049988Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T21:47:35.4050946Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T21:47:35.4051883Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T21:47:35.4052789Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T21:47:35.4053595Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T21:47:35.4054416Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T21:47:35.4099039Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T21:47:35.4099881Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T21:47:35.4100679Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T21:47:35.4101608Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:47:35.4102720Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:47:35.4103771Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T21:47:35.4104841Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:47:35.4105969Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:47:35.4107057Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:47:35.4108300Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:47:35.4109564Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:47:35.4110692Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:47:35.4111817Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:47:35.4113123Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:47:35.4114438Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:47:35.4115733Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:47:35.4117017Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:47:35.4118613Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:47:35.4119936Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:47:35.4121054Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:47:35.4122106Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T21:47:35.4123186Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T21:47:35.4124394Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T21:47:35.4125747Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T21:47:35.4127052Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T21:47:35.4128292Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T21:47:35.4129560Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T21:47:35.4130826Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T21:47:35.4132113Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T21:47:35.4133386Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T21:47:35.4134651Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T21:47:35.4135997Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T21:47:35.4237789Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T21:47:35.4239140Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T21:47:35.4240442Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T21:47:35.4241877Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T21:47:35.4243259Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T21:47:35.4244824Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T21:47:35.4246172Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T21:47:35.4247539Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T21:47:35.4248902Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T21:47:35.4250251Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T21:47:35.4251566Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T21:47:35.4259774Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T21:47:35.4310221Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T21:47:35.4311666Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:47:35.4312912Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:47:35.4314270Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T21:47:35.4315320Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T21:47:35.4316341Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T21:47:35.4317327Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T21:47:35.4318198Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T21:47:35.4319112Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T21:47:35.4414615Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T21:47:35.4454269Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T21:47:35.4454949Z creating: build/lib/ 2024-08-20T21:47:35.4455304Z inflating: build/lib/libclog.a 2024-08-20T21:47:35.4463927Z inflating: build/lib/libpthreadpool.a 2024-08-20T21:47:35.4471647Z inflating: build/lib/libcpuinfo.a 2024-08-20T21:47:35.4479282Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T21:47:35.4540769Z inflating: build/lib/libgtest.a 2024-08-20T21:47:35.4547634Z inflating: build/lib/libittnotify.a 2024-08-20T21:47:35.4625549Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T21:47:35.4695625Z inflating: build/lib/libbenchmark.a 2024-08-20T21:47:35.4721921Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T21:47:35.4782108Z inflating: build/lib/libasmjit.a 2024-08-20T21:47:35.4889283Z inflating: build/lib/libgloo.a 2024-08-20T21:47:35.5302413Z inflating: build/lib/libprotobuf.a 2024-08-20T21:47:35.5322454Z inflating: build/lib/libfmt.a 2024-08-20T21:47:35.5412375Z inflating: build/lib/libc10.so 2024-08-20T21:47:35.5412898Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T21:47:35.5414641Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T21:47:35.5434157Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T21:47:35.5899033Z inflating: build/lib/libprotoc.a 2024-08-20T21:47:35.5915325Z inflating: build/lib/libgmock.a 2024-08-20T21:47:35.5916056Z inflating: build/lib/libgtest_main.a 2024-08-20T21:47:35.5916518Z inflating: build/lib/libbenchmark_main.a 2024-08-20T21:47:35.5933534Z inflating: build/lib/libnnpack.a 2024-08-20T21:47:36.4914969Z inflating: build/lib/libdnnl.a 2024-08-20T21:47:36.5426252Z inflating: build/lib/libtensorpipe.a 2024-08-20T21:47:36.6394299Z inflating: build/lib/libfbgemm.a 2024-08-20T21:47:36.6660485Z inflating: build/lib/libkineto.a 2024-08-20T21:47:36.6660958Z inflating: build/lib/libgmock_main.a 2024-08-20T21:47:36.6699367Z inflating: build/lib/libonnx_proto.a 2024-08-20T21:47:36.6869463Z inflating: build/lib/libXNNPACK.a 2024-08-20T21:47:36.7513055Z inflating: build/lib/libonnx.a 2024-08-20T21:47:38.9463393Z inflating: build/lib/libtorch_cpu.so 2024-08-20T21:47:38.9464285Z inflating: build/lib/libtorch.so 2024-08-20T21:47:38.9468625Z inflating: build/lib/libunbox_lib.a 2024-08-20T21:47:38.9472004Z inflating: build/lib/libshm.so 2024-08-20T21:47:38.9491663Z inflating: build/lib/libjitbackend_test.so 2024-08-20T21:47:38.9558455Z inflating: build/lib/libtorchbind_test.so 2024-08-20T21:47:38.9581394Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T21:47:38.9606007Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T21:47:39.1384600Z inflating: build/lib/libtorch_python.so 2024-08-20T21:47:39.1417903Z inflating: build/lib/libnnapi_backend.so 2024-08-20T21:47:39.1418650Z creating: build/bin/ 2024-08-20T21:47:39.1419006Z creating: build/bin/CMakeFiles/ 2024-08-20T21:47:39.1419602Z inflating: build/bin/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:47:39.1420288Z extracting: build/bin/CMakeFiles/progress.marks 2024-08-20T21:47:39.1421157Z inflating: build/bin/Makefile 2024-08-20T21:47:39.1421613Z inflating: build/bin/cmake_install.cmake 2024-08-20T21:47:39.1422103Z inflating: build/bin/CTestTestfile.cmake 2024-08-20T21:47:39.1471747Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T21:47:39.1522841Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T21:47:39.1574536Z inflating: build/bin/c10_Device_test 2024-08-20T21:47:39.1627123Z inflating: build/bin/c10_Scalar_test 2024-08-20T21:47:39.1688126Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T21:47:39.1736195Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T21:47:39.1786196Z inflating: build/bin/c10_SymInt_test 2024-08-20T21:47:39.1840875Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T21:47:39.1895660Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T21:47:39.1948775Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T21:47:39.1998552Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T21:47:39.2050354Z inflating: build/bin/c10_Bitset_test 2024-08-20T21:47:39.2120446Z inflating: build/bin/c10_cow_test 2024-08-20T21:47:39.2170072Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T21:47:39.2220347Z inflating: build/bin/c10_Half_test 2024-08-20T21:47:39.2277768Z inflating: build/bin/c10_LeftRight_test 2024-08-20T21:47:39.2331947Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T21:47:39.2379797Z inflating: build/bin/c10_Synchronized_test 2024-08-20T21:47:39.2434804Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T21:47:39.2486486Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T21:47:39.2536748Z inflating: build/bin/c10_TypeList_test 2024-08-20T21:47:39.2586993Z inflating: build/bin/c10_accumulate_test 2024-08-20T21:47:39.2637314Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T21:47:39.2692098Z inflating: build/bin/c10_bfloat16_test 2024-08-20T21:47:39.2741463Z inflating: build/bin/c10_bit_cast_test 2024-08-20T21:47:39.2797196Z inflating: build/bin/c10_complex_math_test 2024-08-20T21:47:39.2851630Z inflating: build/bin/c10_complex_test 2024-08-20T21:47:39.2903755Z inflating: build/bin/c10_exception_test 2024-08-20T21:47:39.2953111Z inflating: build/bin/c10_flags_test 2024-08-20T21:47:39.3003529Z inflating: build/bin/c10_generic_math_test 2024-08-20T21:47:39.3058186Z inflating: build/bin/c10_lazy_test 2024-08-20T21:47:39.3105104Z inflating: build/bin/c10_irange_test 2024-08-20T21:47:39.3261986Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T21:47:39.3319467Z inflating: build/bin/c10_logging_test 2024-08-20T21:47:39.3392742Z inflating: build/bin/c10_optional_test 2024-08-20T21:47:39.3444625Z inflating: build/bin/c10_registry_test 2024-08-20T21:47:39.3504694Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T21:47:39.3651507Z inflating: build/bin/c10_small_vector_test 2024-08-20T21:47:39.3701840Z inflating: build/bin/c10_ssize_test 2024-08-20T21:47:39.3753765Z inflating: build/bin/c10_string_util_test 2024-08-20T21:47:39.3811786Z inflating: build/bin/c10_string_view_test 2024-08-20T21:47:39.3861933Z inflating: build/bin/c10_tempfile_test 2024-08-20T21:47:39.3907727Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T21:47:39.3964315Z inflating: build/bin/c10_typeid_test 2024-08-20T21:47:39.4396218Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T21:47:39.4826999Z inflating: build/bin/protoc 2024-08-20T21:47:39.5161803Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T21:47:39.5478287Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T21:47:39.5822520Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T21:47:39.5876583Z inflating: build/bin/BackoffTest 2024-08-20T21:47:39.5928215Z inflating: build/bin/FileStoreTest 2024-08-20T21:47:39.5982549Z inflating: build/bin/TCPStoreTest 2024-08-20T21:47:39.6037117Z inflating: build/bin/HashStoreTest 2024-08-20T21:47:39.6037874Z inflating: build/bin/example_allreduce 2024-08-20T21:47:39.6102808Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T21:47:39.6158540Z inflating: build/bin/test_dist_autograd 2024-08-20T21:47:39.6159467Z inflating: build/bin/parallel_benchmark 2024-08-20T21:47:39.6226134Z inflating: build/bin/test_cpp_rpc 2024-08-20T21:47:39.6234902Z inflating: build/bin/aot_model_compiler_test 2024-08-20T21:47:39.6299731Z inflating: build/bin/test_mobile_nnc 2024-08-20T21:47:39.6624525Z inflating: build/bin/test_lazy 2024-08-20T21:47:39.6682155Z inflating: build/bin/static_runtime_bench 2024-08-20T21:47:39.6752951Z inflating: build/bin/Dict_test 2024-08-20T21:47:39.6805862Z inflating: build/bin/Dimname_test 2024-08-20T21:47:39.7043949Z inflating: build/bin/static_runtime_test 2024-08-20T21:47:39.7099698Z inflating: build/bin/NamedTensor_test 2024-08-20T21:47:39.7164487Z inflating: build/bin/MaybeOwned_test 2024-08-20T21:47:39.7221396Z inflating: build/bin/apply_utils_test 2024-08-20T21:47:39.7281304Z inflating: build/bin/atest 2024-08-20T21:47:39.7342634Z inflating: build/bin/basic 2024-08-20T21:47:39.7397878Z inflating: build/bin/broadcast_test 2024-08-20T21:47:39.7447908Z inflating: build/bin/cpu_allocator_test 2024-08-20T21:47:39.7505690Z inflating: build/bin/cpu_generator_test 2024-08-20T21:47:39.7558063Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T21:47:39.7648745Z inflating: build/bin/cpu_rng_test 2024-08-20T21:47:39.7699101Z inflating: build/bin/dispatch_key_set_test 2024-08-20T21:47:39.7748387Z inflating: build/bin/dlconvertor_test 2024-08-20T21:47:39.7805430Z inflating: build/bin/extension_backend_test 2024-08-20T21:47:39.7859785Z inflating: build/bin/half_test 2024-08-20T21:47:39.8930888Z inflating: build/bin/test_api 2024-08-20T21:47:39.8979659Z inflating: build/bin/lazy_tensor_test 2024-08-20T21:47:39.9032659Z inflating: build/bin/math_kernel_test 2024-08-20T21:47:39.9124125Z inflating: build/bin/ivalue_test 2024-08-20T21:47:39.9177322Z inflating: build/bin/memory_format_test 2024-08-20T21:47:39.9226826Z inflating: build/bin/operator_name_test 2024-08-20T21:47:39.9281300Z inflating: build/bin/memory_overlapping_test 2024-08-20T21:47:39.9337115Z inflating: build/bin/native_test 2024-08-20T21:47:39.9388378Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T21:47:39.9439669Z inflating: build/bin/operators_test 2024-08-20T21:47:39.9491585Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T21:47:39.9557729Z inflating: build/bin/pow_test 2024-08-20T21:47:39.9605766Z inflating: build/bin/reduce_ops_test 2024-08-20T21:47:39.9662528Z inflating: build/bin/quantized_test 2024-08-20T21:47:39.9714911Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T21:47:39.9771442Z inflating: build/bin/scalar_tensor_test 2024-08-20T21:47:39.9821930Z inflating: build/bin/StorageUtils_test 2024-08-20T21:47:39.9881339Z inflating: build/bin/scalar_test 2024-08-20T21:47:39.9932854Z inflating: build/bin/stride_properties_test 2024-08-20T21:47:40.0009391Z inflating: build/bin/tensor_iterator_test 2024-08-20T21:47:40.0063013Z inflating: build/bin/type_ptr_test 2024-08-20T21:47:40.0116307Z inflating: build/bin/test_parallel 2024-08-20T21:47:40.0117484Z inflating: build/bin/thread_init_test 2024-08-20T21:47:40.0178423Z inflating: build/bin/type_test 2024-08-20T21:47:40.0228917Z inflating: build/bin/undefined_tensor_test 2024-08-20T21:47:40.0229836Z inflating: build/bin/verify_api_visibility 2024-08-20T21:47:40.0299492Z inflating: build/bin/legacy_vmap_test 2024-08-20T21:47:40.0349856Z inflating: build/bin/weakref_test 2024-08-20T21:47:40.0402348Z inflating: build/bin/wrapdim_test 2024-08-20T21:47:40.0460120Z inflating: build/bin/IListRef_test 2024-08-20T21:47:40.0511806Z inflating: build/bin/xla_tensor_test 2024-08-20T21:47:40.0613779Z inflating: build/bin/List_test 2024-08-20T21:47:40.0730532Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T21:47:40.0822834Z inflating: build/bin/kernel_function_test 2024-08-20T21:47:40.0945184Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T21:47:40.1009667Z inflating: build/bin/KernelFunction_test 2024-08-20T21:47:40.1107187Z inflating: build/bin/kernel_lambda_test 2024-08-20T21:47:40.1159511Z inflating: build/bin/CppSignature_test 2024-08-20T21:47:40.1218905Z inflating: build/bin/kernel_stackbased_test 2024-08-20T21:47:40.1312657Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T21:47:40.1361316Z inflating: build/bin/op_allowlist_test 2024-08-20T21:47:40.1422390Z inflating: build/bin/inline_container_test 2024-08-20T21:47:40.1716104Z inflating: build/bin/op_registration_test 2024-08-20T21:47:40.1771082Z inflating: build/bin/backend_fallback_test 2024-08-20T21:47:40.1823354Z inflating: build/bin/test_edge_op_registration 2024-08-20T21:47:40.1837354Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T21:47:40.1839261Z inflating: build/bin/torch_shm_manager 2024-08-20T21:47:40.2591742Z inflating: build/bin/test_tensorexpr 2024-08-20T21:47:40.3142694Z inflating: build/bin/test_jit 2024-08-20T21:47:40.3143160Z creating: .additional_ci_files/ 2024-08-20T21:47:40.3200023Z inflating: .additional_ci_files/test-times.json 2024-08-20T21:47:40.3423438Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T21:47:40.3459774Z ##[group]Run rm artifacts.zip 2024-08-20T21:47:40.3460158Z rm artifacts.zip 2024-08-20T21:47:40.3465801Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:40.3466297Z env: 2024-08-20T21:47:40.3466548Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:40.3466883Z ##[endgroup] 2024-08-20T21:47:40.3850189Z ##[group]Run df -H 2024-08-20T21:47:40.3850495Z df -H 2024-08-20T21:47:40.3855977Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:40.3856466Z env: 2024-08-20T21:47:40.3856725Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:40.3857052Z ##[endgroup] 2024-08-20T21:47:40.3915474Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T21:47:40.3916106Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T21:47:40.3916586Z tmpfs 50G 0 50G 0% /dev/shm 2024-08-20T21:47:40.3917110Z tmpfs 20G 652k 20G 1% /run 2024-08-20T21:47:40.3917673Z /dev/nvme0n1p1 215G 39G 177G 18% / 2024-08-20T21:47:40.3918118Z tmpfs 50G 4.1k 50G 1% /tmp 2024-08-20T21:47:40.3918584Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T21:47:40.3959608Z Prepare all required actions 2024-08-20T21:47:40.3960032Z Getting action download info 2024-08-20T21:47:40.5078929Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T21:47:40.5079384Z with: 2024-08-20T21:47:40.5079647Z env: 2024-08-20T21:47:40.5079910Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:40.5080246Z ##[endgroup] 2024-08-20T21:47:40.5313396Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T21:47:40.5313840Z with: 2024-08-20T21:47:40.5314083Z name: td_results 2024-08-20T21:47:40.5314370Z s3-bucket: gha-artifacts 2024-08-20T21:47:40.5314704Z region: us-east-1 2024-08-20T21:47:40.5314981Z env: 2024-08-20T21:47:40.5315231Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:40.5315540Z ##[endgroup] 2024-08-20T21:47:40.9942745Z (node:43736) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T21:47:40.9943680Z 2024-08-20T21:47:40.9944016Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T21:47:40.9944762Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T21:47:40.9945661Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T21:47:41.0693745Z Found 0 objects with prefix pytorch/pytorch/10479310725/td_results/ 2024-08-20T21:47:41.0698801Z Artifact download has finished successfully 2024-08-20T21:47:41.0902176Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T21:47:41.0902647Z mkdir -p .additional_ci_files 2024-08-20T21:47:41.0903208Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T21:47:41.0909333Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:41.0909815Z env: 2024-08-20T21:47:41.0910079Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:41.0910411Z ##[endgroup] 2024-08-20T21:47:41.0956735Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-20T21:47:41.0968423Z ##[error]Process completed with exit code 1. 2024-08-20T21:47:41.1626544Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T21:47:41.1627085Z .github/scripts/parse_ref.py 2024-08-20T21:47:41.1632589Z shell: /usr/bin/bash -e {0} 2024-08-20T21:47:41.1632904Z env: 2024-08-20T21:47:41.1633149Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:41.1633467Z ##[endgroup] 2024-08-20T21:47:41.1903506Z Prepare all required actions 2024-08-20T21:47:41.1951778Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T21:47:41.1952234Z with: 2024-08-20T21:47:41.1952783Z github-token: *** 2024-08-20T21:47:41.1953072Z env: 2024-08-20T21:47:41.1953312Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:41.1953643Z ##[endgroup] 2024-08-20T21:47:41.2159065Z ##[group]Run set -eux 2024-08-20T21:47:41.2159384Z set -eux 2024-08-20T21:47:41.2159965Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T21:47:41.2165500Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:41.2165957Z env: 2024-08-20T21:47:41.2166202Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:41.2166683Z GITHUB_TOKEN: *** 2024-08-20T21:47:41.2166946Z ##[endgroup] 2024-08-20T21:47:41.2188370Z + python3 .github/scripts/get_workflow_job_id.py 10479310725 i-0f00d0841bbe75141 2024-08-20T21:47:42.2903532Z setting job-id=29025551933 2024-08-20T21:47:42.2904774Z setting job-name=linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_amp_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:47:42.3159573Z Prepare all required actions 2024-08-20T21:47:42.3160011Z Getting action download info 2024-08-20T21:47:42.4302915Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T21:47:42.4303367Z with: 2024-08-20T21:47:42.4303833Z github-token: *** 2024-08-20T21:47:42.4323655Z test-matrix: {"include": [{"config": "inductor_avx512", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_avx512", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_amp_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_torchbench_cpu_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.24xl.spr-metal"}, {"config": "inductor_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "inductor_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_huggingface_freezing_avx2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}]} 2024-08-20T21:47:42.4344858Z job-name: linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_amp_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:47:42.4345802Z env: 2024-08-20T21:47:42.4346074Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:42.4346402Z ##[endgroup] 2024-08-20T21:47:42.4544288Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T21:47:42.4544851Z with: 2024-08-20T21:47:42.4545106Z shell: bash 2024-08-20T21:47:42.4545389Z timeout_minutes: 10 2024-08-20T21:47:42.4545699Z max_attempts: 5 2024-08-20T21:47:42.4545985Z retry_wait_seconds: 30 2024-08-20T21:47:42.4547159Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T21:47:42.4548390Z polling_interval_seconds: 1 2024-08-20T21:47:42.4548760Z warning_on_retry: true 2024-08-20T21:47:42.4549256Z continue_on_error: false 2024-08-20T21:47:42.4549589Z env: 2024-08-20T21:47:42.4549855Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:42.4550405Z GITHUB_TOKEN: *** 2024-08-20T21:47:42.4550692Z ##[endgroup] 2024-08-20T21:47:42.5248822Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T21:47:42.7600869Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T21:47:42.9728367Z Collecting requests==2.27.1 2024-08-20T21:47:43.0102343Z Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB) 2024-08-20T21:47:43.2784087Z Collecting pyyaml==6.0.1 2024-08-20T21:47:43.2840591Z Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (738 kB) 2024-08-20T21:47:43.6923213Z Collecting charset-normalizer~=2.0.0 2024-08-20T21:47:43.6978075Z Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2024-08-20T21:47:43.7531658Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-20T21:47:43.7540430Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-20T21:47:43.8559639Z Collecting certifi>=2017.4.17 2024-08-20T21:47:43.8613486Z Downloading certifi-2024.7.4-py3-none-any.whl (162 kB) 2024-08-20T21:47:43.9874170Z Installing collected packages: charset-normalizer, certifi, requests, pyyaml 2024-08-20T21:47:44.4355182Z Successfully installed certifi-2024.7.4 charset-normalizer-2.0.12 pyyaml-6.0.1 requests-2.27.1 2024-08-20T21:47:45.5096883Z Command completed after 1 attempt(s). 2024-08-20T21:47:45.5199655Z ##[group]Run set -x 2024-08-20T21:47:45.5199972Z set -x 2024-08-20T21:47:45.5200250Z  2024-08-20T21:47:45.5200780Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T21:47:45.5201427Z # in runner workspace 2024-08-20T21:47:45.5201929Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T21:47:45.5208347Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:45.5208824Z env: 2024-08-20T21:47:45.5209254Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:45.5209591Z ##[endgroup] 2024-08-20T21:47:45.5239671Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T21:47:45.5579596Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T21:47:45.5580198Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T21:47:45.5580646Z echo "Job name: ${JOB_NAME}" 2024-08-20T21:47:45.5581041Z  2024-08-20T21:47:45.5581594Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T21:47:45.5582295Z # in runner workspace 2024-08-20T21:47:45.5582862Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T21:47:45.5583518Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T21:47:45.5583987Z  --job-name "${JOB_NAME}" \ 2024-08-20T21:47:45.5604884Z  --test-matrix "{"include": [{"config": "inductor_avx512", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_avx512", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_amp_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_torchbench_cpu_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.24xl.spr-metal"}, {"config": "inductor_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "inductor_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_huggingface_freezing_avx2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}]}" \ 2024-08-20T21:47:45.5626321Z  --selected-test-configs "" \ 2024-08-20T21:47:45.5626787Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T21:47:45.5627212Z  --tag "${TAG}" \ 2024-08-20T21:47:45.5627583Z  --event-name "${EVENT_NAME}" \ 2024-08-20T21:47:45.5628031Z  --schedule "${SCHEDULE}" \ 2024-08-20T21:47:45.5628525Z  --branch "${HEAD_BRANCH}" 2024-08-20T21:47:45.5634437Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:45.5634950Z env: 2024-08-20T21:47:45.5635213Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:45.5635760Z GITHUB_TOKEN: *** 2024-08-20T21:47:45.5636607Z JOB_NAME: linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_amp_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:47:45.5637527Z PR_NUMBER: 2024-08-20T21:47:45.5637824Z TAG: ciflow/inductor/133712 2024-08-20T21:47:45.5638177Z EVENT_NAME: push 2024-08-20T21:47:45.5638455Z SCHEDULE: 2024-08-20T21:47:45.5638722Z HEAD_BRANCH: 2024-08-20T21:47:45.5639000Z ##[endgroup] 2024-08-20T21:47:45.5664351Z Workflow: inductor 2024-08-20T21:47:45.5665529Z Job name: linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_amp_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:47:45.8353348Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T21:47:45.9849650Z ##[group]Run echo "Filtered matrix:" 2024-08-20T21:47:45.9850234Z echo "Filtered matrix:" 2024-08-20T21:47:45.9870771Z echo "{"include": [{"config": "inductor_avx512", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_avx512", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_inductor_huggingface_amp_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_timm_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "cpu_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.16xlarge.spr"}, {"config": "dynamic_cpu_inductor_huggingface", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_timm", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_inductor_torchbench", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_huggingface_freezing", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_timm_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "dynamic_cpu_aot_inductor_torchbench_amp_freezing", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.12xlarge"}, {"config": "inductor_torchbench_cpu_smoketest_perf", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.24xl.spr-metal"}, {"config": "inductor_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "inductor_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_huggingface_freezing_avx2", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_torchbench_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}, {"config": "cpu_inductor_timm_freezing_avx2", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.10xlarge.avx2"}]}" 2024-08-20T21:47:45.9891240Z  2024-08-20T21:47:45.9891491Z echo 2024-08-20T21:47:45.9891867Z echo "Is the current job unstable? False" 2024-08-20T21:47:45.9892330Z  2024-08-20T21:47:45.9892578Z echo 2024-08-20T21:47:45.9892923Z echo "Is keep-going label set? False" 2024-08-20T21:47:45.9893371Z  2024-08-20T21:47:45.9893682Z echo 2024-08-20T21:47:45.9893974Z echo "Renabled issues? " 2024-08-20T21:47:45.9899869Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:45.9900374Z env: 2024-08-20T21:47:45.9900644Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:45.9900964Z ##[endgroup] 2024-08-20T21:47:45.9925128Z Filtered matrix: 2024-08-20T21:47:45.9952990Z {include: [{config: inductor_avx512, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: inductor_avx512, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_huggingface, shard: 1, num_shards: 1, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_torchbench, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_torchbench, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_huggingface_freezing, shard: 1, num_shards: 1, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_timm_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_timm_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_torchbench_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_torchbench_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_inductor_huggingface_amp_freezing, shard: 1, num_shards: 1, runner: amz2023.linux.16xlarge.spr}, {config: cpu_inductor_timm_amp_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.16xlarge.spr}, {config: cpu_inductor_timm_amp_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.16xlarge.spr}, {config: cpu_inductor_torchbench_amp_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.16xlarge.spr}, {config: cpu_inductor_torchbench_amp_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.16xlarge.spr}, {config: dynamic_cpu_inductor_huggingface, shard: 1, num_shards: 1, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_inductor_timm, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_inductor_timm, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_inductor_torchbench, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_inductor_torchbench, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_huggingface_freezing, shard: 1, num_shards: 1, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_timm_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_timm_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_torchbench_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_torchbench_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_torchbench_amp_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: cpu_aot_inductor_torchbench_amp_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_aot_inductor_torchbench_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_aot_inductor_torchbench_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_aot_inductor_torchbench_amp_freezing, shard: 1, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: dynamic_cpu_aot_inductor_torchbench_amp_freezing, shard: 2, num_shards: 2, runner: amz2023.linux.12xlarge}, {config: inductor_torchbench_cpu_smoketest_perf, shard: 1, num_shards: 1, runner: amz2023.linux.24xl.spr-metal}, {config: inductor_avx2, shard: 1, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}, {config: inductor_avx2, shard: 2, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}, {config: cpu_inductor_huggingface_freezing_avx2, shard: 1, num_shards: 1, runner: amz2023.linux.10xlarge.avx2}, {config: cpu_inductor_torchbench_freezing_avx2, shard: 1, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}, {config: cpu_inductor_torchbench_freezing_avx2, shard: 2, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}, {config: cpu_inductor_timm_freezing_avx2, shard: 1, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}, {config: cpu_inductor_timm_freezing_avx2, shard: 2, num_shards: 2, runner: amz2023.linux.10xlarge.avx2}]} 2024-08-20T21:47:45.9972855Z 2024-08-20T21:47:45.9973003Z Is the current job unstable? False 2024-08-20T21:47:45.9973277Z 2024-08-20T21:47:45.9973487Z Is keep-going label set? False 2024-08-20T21:47:45.9973733Z 2024-08-20T21:47:45.9973857Z Renabled issues? 2024-08-20T21:47:46.0180608Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T21:47:46.0181354Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T21:47:46.0186945Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:47:46.0187398Z env: 2024-08-20T21:47:46.0187651Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:46.0187973Z JOB_TIMEOUT: 240 2024-08-20T21:47:46.0188250Z ##[endgroup] 2024-08-20T21:47:46.0472127Z ##[group]Run set -x 2024-08-20T21:47:46.0472499Z set -x 2024-08-20T21:47:46.0472767Z  2024-08-20T21:47:46.0473084Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T21:47:46.0473764Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T21:47:46.0474315Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T21:47:46.0474815Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T21:47:46.0475209Z else 2024-08-20T21:47:46.0475516Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T21:47:46.0475925Z fi 2024-08-20T21:47:46.0476176Z  2024-08-20T21:47:46.0476748Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T21:47:46.0477492Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T21:47:46.0478161Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T21:47:46.0478762Z # shellcheck disable=SC2086,SC2090 2024-08-20T21:47:46.0479199Z container_name=$(docker run \ 2024-08-20T21:47:46.0479604Z  ${GPU_FLAG:-} \ 2024-08-20T21:47:46.0480065Z  -e BUILD_ENVIRONMENT \ 2024-08-20T21:47:46.0480433Z  -e PR_NUMBER \ 2024-08-20T21:47:46.0480745Z  -e GITHUB_ACTIONS \ 2024-08-20T21:47:46.0481104Z  -e GITHUB_REPOSITORY \ 2024-08-20T21:47:46.0481479Z  -e GITHUB_WORKFLOW \ 2024-08-20T21:47:46.0481832Z  -e GITHUB_JOB \ 2024-08-20T21:47:46.0482146Z  -e GITHUB_RUN_ID \ 2024-08-20T21:47:46.0482495Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T21:47:46.0482864Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T21:47:46.0483230Z  -e JOB_ID \ 2024-08-20T21:47:46.0483520Z  -e JOB_NAME \ 2024-08-20T21:47:46.0483831Z  -e BASE_SHA \ 2024-08-20T21:47:46.0484140Z  -e BRANCH \ 2024-08-20T21:47:46.0484424Z  -e SHA1 \ 2024-08-20T21:47:46.0484733Z  -e AWS_DEFAULT_REGION \ 2024-08-20T21:47:46.0485102Z  -e IN_WHEEL_TEST \ 2024-08-20T21:47:46.0485448Z  -e SHARD_NUMBER \ 2024-08-20T21:47:46.0485769Z  -e TEST_CONFIG \ 2024-08-20T21:47:46.0486109Z  -e NUM_TEST_SHARDS \ 2024-08-20T21:47:46.0486466Z  -e REENABLED_ISSUES \ 2024-08-20T21:47:46.0486849Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T21:47:46.0487227Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T21:47:46.0487592Z  -e TEST_SHOWLOCALS \ 2024-08-20T21:47:46.0487949Z  -e NO_TEST_TIMEOUT \ 2024-08-20T21:47:46.0488289Z  -e NO_TD \ 2024-08-20T21:47:46.0488580Z  -e TD_DISTRIBUTED \ 2024-08-20T21:47:46.0488932Z  -e PR_LABELS \ 2024-08-20T21:47:46.0489302Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T21:47:46.0489708Z  -e SCCACHE_BUCKET \ 2024-08-20T21:47:46.0490069Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T21:47:46.0490443Z  -e XLA_CUDA \ 2024-08-20T21:47:46.0490807Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T21:47:46.0491272Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T21:47:46.0491758Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T21:47:46.0492241Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T21:47:46.0492676Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T21:47:46.0493079Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T21:47:46.0493487Z  -e DASHBOARD_TAG \ 2024-08-20T21:47:46.0493929Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T21:47:46.0494448Z  --security-opt seccomp=unconfined \ 2024-08-20T21:47:46.0494872Z  --cap-add=SYS_PTRACE \ 2024-08-20T21:47:46.0495228Z  --ipc=host \ 2024-08-20T21:47:46.0495552Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T21:47:46.0495911Z  --tty \ 2024-08-20T21:47:46.0496181Z  --detach \ 2024-08-20T21:47:46.0496621Z  --name="${container_name}" \ 2024-08-20T21:47:46.0497016Z  --user jenkins \ 2024-08-20T21:47:46.0497465Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T21:47:46.0497973Z  -w /var/lib/jenkins/workspace \ 2024-08-20T21:47:46.0498673Z  "${DOCKER_IMAGE}" 2024-08-20T21:47:46.0499013Z ) 2024-08-20T21:47:46.0499465Z # Propagate download.pytorch.org IP to container 2024-08-20T21:47:46.0500414Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T21:47:46.0501432Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T21:47:46.0502453Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T21:47:46.0508045Z shell: /usr/bin/bash -e {0} 2024-08-20T21:47:46.0508346Z env: 2024-08-20T21:47:46.0508592Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:47:46.0508997Z BUILD_ENVIRONMENT: linux-jammy-py3.8-gcc11-build 2024-08-20T21:47:46.0509430Z PR_NUMBER: 2024-08-20T21:47:46.0509709Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T21:47:46.0510093Z GITHUB_WORKFLOW: inductor 2024-08-20T21:47:46.0510411Z GITHUB_JOB: test 2024-08-20T21:47:46.0510694Z GITHUB_RUN_ID: 10479310725 2024-08-20T21:47:46.0511006Z GITHUB_RUN_NUMBER: 88082 2024-08-20T21:47:46.0511322Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T21:47:46.0511626Z JOB_ID: 29025551933 2024-08-20T21:47:46.0512430Z JOB_NAME: linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_amp_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:47:46.0513275Z BRANCH: 2024-08-20T21:47:46.0513588Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:47:46.0514071Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:47:46.0514623Z TEST_CONFIG: dynamic_cpu_aot_inductor_torchbench_amp_freezing 2024-08-20T21:47:46.0515091Z SHARD_NUMBER: 1 2024-08-20T21:47:46.0515372Z NUM_TEST_SHARDS: 2 2024-08-20T21:47:46.0515661Z REENABLED_ISSUES: 2024-08-20T21:47:46.0515963Z CONTINUE_THROUGH_ERROR: False 2024-08-20T21:47:46.0516300Z VERBOSE_TEST_LOGS: False 2024-08-20T21:47:46.0516620Z TEST_SHOWLOCALS: False 2024-08-20T21:47:46.0516933Z NO_TEST_TIMEOUT: False 2024-08-20T21:47:46.0517216Z NO_TD: False 2024-08-20T21:47:46.0517482Z TD_DISTRIBUTED: False 2024-08-20T21:47:46.0517865Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T21:47:46.0518323Z SCCACHE_S3_KEY_PREFIX: inductor 2024-08-20T21:47:46.0518653Z SHM_SIZE: 1g 2024-08-20T21:47:46.0519630Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:47:46.0520700Z XLA_CUDA: 2024-08-20T21:47:46.0521151Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:47:46.0521721Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T21:47:46.0522119Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T21:47:46.0522494Z DASHBOARD_TAG: 2024-08-20T21:47:46.0522988Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-20T21:47:46.0523337Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T21:47:46.0523661Z ##[endgroup] 2024-08-20T21:47:46.0547020Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == \m\u\l\t\i\g\p\u ]] 2024-08-20T21:47:46.0547883Z + [[ linux-jammy-py3.8-gcc11-build == *onnx* ]] 2024-08-20T21:47:46.0548375Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T21:47:46.0558435Z +++ nproc --ignore=2 2024-08-20T21:47:46.0832632Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=46 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310725 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:49:00.3511015Z + container_name=8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T21:49:00.3516304Z + grep download.pytorch.org /etc/hosts 2024-08-20T21:49:00.3518464Z + docker exec -i 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T21:49:00.4799701Z + echo DOCKER_CONTAINER_ID=8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T21:49:00.4803724Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl 2024-08-20T21:49:00.4806951Z + docker exec -t 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T21:49:00.8510706Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T21:49:01.2768577Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T21:49:01.2770737Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T21:49:01.2772845Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T21:49:01.2775325Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T21:49:01.2778938Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T21:49:01.2784869Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.12.1) 2024-08-20T21:49:01.2802951Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T21:49:01.2818129Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T21:49:01.2836814Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.21.2) 2024-08-20T21:49:01.3204623Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T21:49:01.9701486Z Installing collected packages: torch 2024-08-20T21:49:09.3662737Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T21:49:09.3664249Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-20T21:49:09.3664971Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T21:49:09.4292333Z + export TERM=vt100 2024-08-20T21:49:09.4292745Z + TERM=vt100 2024-08-20T21:49:09.4293044Z ++ dirname .ci/pytorch/test.sh 2024-08-20T21:49:09.4298674Z + source .ci/pytorch/common.sh 2024-08-20T21:49:09.4301342Z +++ dirname .ci/pytorch/common.sh 2024-08-20T21:49:09.4307117Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T21:49:09.4308524Z +++ declare -f -t trap_add 2024-08-20T21:49:09.4311724Z ++ set -ex 2024-08-20T21:49:09.4312112Z ++ [[ linux-jammy-py3.8-gcc11-build == *rocm* ]] 2024-08-20T21:49:09.4312579Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T21:49:09.4313029Z + [[ linux-jammy-py3.8-gcc11-build != *rocm* ]] 2024-08-20T21:49:09.4315864Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T21:49:09.4325052Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T21:49:09.4325477Z + trap_add cleanup_workspace EXIT 2024-08-20T21:49:09.4325870Z + trap_add_cmd=cleanup_workspace 2024-08-20T21:49:09.4326226Z + shift 2024-08-20T21:49:09.4326473Z + for trap_add_name in "$@" 2024-08-20T21:49:09.4332770Z +++ trap -p EXIT 2024-08-20T21:49:09.4334964Z ++ eval 'extract_trap_cmd ' 2024-08-20T21:49:09.4335310Z +++ extract_trap_cmd 2024-08-20T21:49:09.4335637Z +++ printf '%s\n' '' 2024-08-20T21:49:09.4336014Z ++ printf '%s\n' cleanup_workspace 2024-08-20T21:49:09.4336818Z + trap -- ' 2024-08-20T21:49:09.4337125Z cleanup_workspace' EXIT 2024-08-20T21:49:09.4337568Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T21:49:09.8658619Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T21:49:09.8670461Z + echo 'Environment variables:' 2024-08-20T21:49:09.8670867Z Environment variables: 2024-08-20T21:49:09.8671177Z + env 2024-08-20T21:49:09.8679248Z INSTALLED_DB=yes 2024-08-20T21:49:09.8679976Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:49:09.8680580Z CONTINUE_THROUGH_ERROR=False 2024-08-20T21:49:09.8681064Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11-build 2024-08-20T21:49:09.8681513Z HOSTNAME=8f9765e7049f 2024-08-20T21:49:09.8682415Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_cc3f7868-0b53-432e-9789-e9f0fb098405 2024-08-20T21:49:09.8683250Z GITHUB_ACTION=__self 2024-08-20T21:49:09.8683587Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T21:49:09.8683962Z GITHUB_RUN_NUMBER=88082 2024-08-20T21:49:09.8684410Z TEST_CONFIG=dynamic_cpu_aot_inductor_torchbench_amp_freezing 2024-08-20T21:49:09.8684950Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T21:49:09.8685423Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T21:49:09.8685828Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T21:49:09.8686269Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T21:49:09.8686678Z GITHUB_REF_TYPE=tag 2024-08-20T21:49:09.8686975Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T21:49:09.8687398Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:09.8687840Z XLA_CUDA= 2024-08-20T21:49:09.8688337Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T21:49:09.8693265Z *** 2024-08-20T21:49:09.8693519Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T21:49:09.8693882Z GITHUB_ACTIONS=true 2024-08-20T21:49:09.8711117Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:09.8711653Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:09.8712488Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T21:49:09.8713226Z UCC_HOME=/usr 2024-08-20T21:49:09.8713503Z VERBOSE_TEST_LOGS=False 2024-08-20T21:49:09.8713895Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T21:49:09.8714298Z SHARD_NUMBER=1 2024-08-20T21:49:09.8714588Z GITHUB_REF_PROTECTED=false 2024-08-20T21:49:09.8714925Z HOME=/var/lib/jenkins 2024-08-20T21:49:09.8715276Z GITHUB_API_URL=https://api.github.com 2024-08-20T21:49:09.8715705Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T21:49:09.8716083Z UCX_COMMIT= 2024-08-20T21:49:09.8716357Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T21:49:09.8716698Z NUM_TEST_SHARDS=2 2024-08-20T21:49:09.8716978Z UCX_HOME=/usr 2024-08-20T21:49:09.8718103Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_cc3f7868-0b53-432e-9789-e9f0fb098405 2024-08-20T21:49:09.8719649Z JOB_NAME=linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_amp_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:49:09.8721140Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_cc3f7868-0b53-432e-9789-e9f0fb098405 2024-08-20T21:49:09.8722308Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T21:49:09.8722984Z GITHUB_EVENT_NAME=push 2024-08-20T21:49:09.8723297Z DASHBOARD_TAG= 2024-08-20T21:49:09.8723562Z GITHUB_RUN_ID=10479310725 2024-08-20T21:49:09.8724521Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_cc3f7868-0b53-432e-9789-e9f0fb098405 2024-08-20T21:49:09.8725621Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T21:49:09.8725971Z PR_NUMBER= 2024-08-20T21:49:09.8726215Z DESIRED_CUDA= 2024-08-20T21:49:09.8726494Z GITHUB_RUN_ATTEMPT=1 2024-08-20T21:49:09.8726818Z ANACONDA_PYTHON_VERSION=3.8 2024-08-20T21:49:09.8727249Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T21:49:09.8727694Z TERM=vt100 2024-08-20T21:49:09.8727955Z INSTALLED_VISION=yes 2024-08-20T21:49:09.8728244Z BRANCH= 2024-08-20T21:49:09.8728498Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T21:49:09.8728855Z CUDA_PATH=/usr/local/cuda 2024-08-20T21:49:09.8729636Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T21:49:09.8730412Z GITHUB_SERVER_URL=https://github.com 2024-08-20T21:49:09.8730787Z UCC_COMMIT= 2024-08-20T21:49:09.8731050Z REENABLED_ISSUES= 2024-08-20T21:49:09.8731328Z DOCS=yes 2024-08-20T21:49:09.8731587Z INSTALLED_ANDROID= 2024-08-20T21:49:09.8731855Z SHLVL=1 2024-08-20T21:49:09.8732092Z MAX_JOBS=46 2024-08-20T21:49:09.8732357Z GITHUB_ACTOR_ID=54816060 2024-08-20T21:49:09.8732792Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:09.8733332Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T21:49:09.8734004Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:49:09.8734580Z GITHUB_JOB=test 2024-08-20T21:49:09.8734851Z NO_TEST_TIMEOUT=False 2024-08-20T21:49:09.8735159Z TD_DISTRIBUTED=False 2024-08-20T21:49:09.8735486Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T21:49:09.8735871Z GITHUB_RETENTION_DAYS=90 2024-08-20T21:49:09.8736189Z OPENSSL_DIR=/opt/openssl 2024-08-20T21:49:09.8736519Z GITHUB_ACTION_REPOSITORY= 2024-08-20T21:49:09.8737604Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:49:09.8738720Z GITHUB_BASE_REF= 2024-08-20T21:49:09.8738996Z INSTALLED_ACL= 2024-08-20T21:49:09.8739386Z CI=true 2024-08-20T21:49:09.8739662Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T21:49:09.8740012Z JOB_ID=29025551933 2024-08-20T21:49:09.8740306Z INSTALLED_PROTOBUF=yes 2024-08-20T21:49:09.8740619Z GITHUB_HEAD_REF= 2024-08-20T21:49:09.8740907Z GITHUB_ACTION_REF= 2024-08-20T21:49:09.8741339Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T21:49:09.8741801Z TEST_SHOWLOCALS=False 2024-08-20T21:49:09.8742118Z GITHUB_WORKFLOW=inductor 2024-08-20T21:49:09.8742450Z DEBIAN_FRONTEND=noninteractive 2024-08-20T21:49:09.8743599Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_cc3f7868-0b53-432e-9789-e9f0fb098405 2024-08-20T21:49:09.8744449Z NO_TD=False 2024-08-20T21:49:09.8744735Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T21:49:09.8745072Z _=/usr/bin/env 2024-08-20T21:49:09.8745548Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T21:49:09.8906144Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch 2024-08-20T21:49:09.8907122Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-20T21:49:09.8908015Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib 2024-08-20T21:49:09.8909079Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/test 2024-08-20T21:49:09.8909817Z + BUILD_DIR=build 2024-08-20T21:49:09.8910132Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T21:49:09.8910512Z + BUILD_BIN_DIR=build/bin 2024-08-20T21:49:09.8910825Z + SHARD_NUMBER=1 2024-08-20T21:49:09.8911109Z + NUM_TEST_SHARDS=2 2024-08-20T21:49:09.8911407Z + export VALGRIND=ON 2024-08-20T21:49:09.8911695Z + VALGRIND=ON 2024-08-20T21:49:09.8912114Z + [[ linux-jammy-py3.8-gcc11-build == *clang9* ]] 2024-08-20T21:49:09.8912565Z + [[ 0 == \1 ]] 2024-08-20T21:49:09.8912837Z + [[ False == \1 ]] 2024-08-20T21:49:09.8913245Z + [[ linux-jammy-py3.8-gcc11-build != *bazel* ]] 2024-08-20T21:49:09.8913735Z ++ realpath build/custom_test_artifacts 2024-08-20T21:49:09.8916444Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T21:49:09.8917149Z + [[ -n '' ]] 2024-08-20T21:49:09.8917469Z + echo 'Environment variables' 2024-08-20T21:49:09.8917837Z Environment variables 2024-08-20T21:49:09.8918132Z + env 2024-08-20T21:49:09.8922539Z INSTALLED_DB=yes 2024-08-20T21:49:09.8923435Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:49:09.8924228Z CONTINUE_THROUGH_ERROR=False 2024-08-20T21:49:09.8924747Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11-build 2024-08-20T21:49:09.8925246Z HOSTNAME=8f9765e7049f 2024-08-20T21:49:09.8926128Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_cc3f7868-0b53-432e-9789-e9f0fb098405 2024-08-20T21:49:09.8926958Z GITHUB_ACTION=__self 2024-08-20T21:49:09.8927289Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T21:49:09.8927819Z GITHUB_RUN_NUMBER=88082 2024-08-20T21:49:09.8928439Z TEST_CONFIG=dynamic_cpu_aot_inductor_torchbench_amp_freezing 2024-08-20T21:49:09.8929153Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T21:49:09.8929639Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T21:49:09.8930064Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T21:49:09.8930488Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T21:49:09.8930904Z GITHUB_REF_TYPE=tag 2024-08-20T21:49:09.8931211Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T21:49:09.8931623Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:09.8932050Z XLA_CUDA= 2024-08-20T21:49:09.8932508Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T21:49:09.8932885Z *** 2024-08-20T21:49:09.8933139Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T21:49:09.8933478Z GITHUB_ACTIONS=true 2024-08-20T21:49:09.8933832Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:09.8934336Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:09.8935159Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/133712 2024-08-20T21:49:09.8935889Z UCC_HOME=/usr 2024-08-20T21:49:09.8936167Z VERBOSE_TEST_LOGS=False 2024-08-20T21:49:09.8936528Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T21:49:09.8936940Z SHARD_NUMBER=1 2024-08-20T21:49:09.8937215Z GITHUB_REF_PROTECTED=false 2024-08-20T21:49:09.8937549Z HOME=/var/lib/jenkins 2024-08-20T21:49:09.8937903Z GITHUB_API_URL=https://api.github.com 2024-08-20T21:49:09.8938322Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T21:49:09.8938703Z UCX_COMMIT= 2024-08-20T21:49:09.8938978Z SCCACHE_S3_KEY_PREFIX=inductor 2024-08-20T21:49:09.8939442Z NUM_TEST_SHARDS=2 2024-08-20T21:49:09.8939712Z UCX_HOME=/usr 2024-08-20T21:49:09.8940580Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_cc3f7868-0b53-432e-9789-e9f0fb098405 2024-08-20T21:49:09.8942102Z JOB_NAME=linux-jammy-cpu-py3.8-gcc11-inductor / test (dynamic_cpu_aot_inductor_torchbench_amp_freezing, 1, 2, amz2023.linux.12xlarge) 2024-08-20T21:49:09.8943771Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_cc3f7868-0b53-432e-9789-e9f0fb098405 2024-08-20T21:49:09.8944960Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T21:49:09.8945635Z GITHUB_EVENT_NAME=push 2024-08-20T21:49:09.8945954Z DASHBOARD_TAG= 2024-08-20T21:49:09.8946239Z GITHUB_RUN_ID=10479310725 2024-08-20T21:49:09.8947334Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_cc3f7868-0b53-432e-9789-e9f0fb098405 2024-08-20T21:49:09.8948298Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T21:49:09.8948649Z PR_NUMBER= 2024-08-20T21:49:09.8948910Z DESIRED_CUDA= 2024-08-20T21:49:09.8949172Z GITHUB_RUN_ATTEMPT=1 2024-08-20T21:49:09.8949472Z VALGRIND=ON 2024-08-20T21:49:09.8949749Z ANACONDA_PYTHON_VERSION=3.8 2024-08-20T21:49:09.8950177Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T21:49:09.8950615Z TERM=vt100 2024-08-20T21:49:09.8950875Z INSTALLED_VISION=yes 2024-08-20T21:49:09.8951165Z BRANCH= 2024-08-20T21:49:09.8951502Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T21:49:09.8951861Z CUDA_PATH=/usr/local/cuda 2024-08-20T21:49:09.8952647Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T21:49:09.8953423Z GITHUB_SERVER_URL=https://github.com 2024-08-20T21:49:09.8953799Z UCC_COMMIT= 2024-08-20T21:49:09.8954065Z REENABLED_ISSUES= 2024-08-20T21:49:09.8954342Z DOCS=yes 2024-08-20T21:49:09.8954584Z INSTALLED_ANDROID= 2024-08-20T21:49:09.8954864Z SHLVL=1 2024-08-20T21:49:09.8955098Z MAX_JOBS=46 2024-08-20T21:49:09.8955362Z GITHUB_ACTOR_ID=54816060 2024-08-20T21:49:09.8955799Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:49:09.8956335Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T21:49:09.8957007Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:49:09.8957583Z GITHUB_JOB=test 2024-08-20T21:49:09.8957852Z NO_TEST_TIMEOUT=False 2024-08-20T21:49:09.8958158Z TD_DISTRIBUTED=False 2024-08-20T21:49:09.8958491Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T21:49:09.8958863Z GITHUB_RETENTION_DAYS=90 2024-08-20T21:49:09.8959194Z OPENSSL_DIR=/opt/openssl 2024-08-20T21:49:09.8959526Z GITHUB_ACTION_REPOSITORY= 2024-08-20T21:49:09.8960612Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:49:09.8961715Z GITHUB_BASE_REF= 2024-08-20T21:49:09.8962020Z INSTALLED_ACL= 2024-08-20T21:49:09.8962284Z CI=true 2024-08-20T21:49:09.8962550Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T21:49:09.8962894Z JOB_ID=29025551933 2024-08-20T21:49:09.8963191Z INSTALLED_PROTOBUF=yes 2024-08-20T21:49:09.8963502Z GITHUB_HEAD_REF= 2024-08-20T21:49:09.8963789Z GITHUB_ACTION_REF= 2024-08-20T21:49:09.8964204Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T21:49:09.8964666Z TEST_SHOWLOCALS=False 2024-08-20T21:49:09.8964982Z GITHUB_WORKFLOW=inductor 2024-08-20T21:49:09.8965324Z DEBIAN_FRONTEND=noninteractive 2024-08-20T21:49:09.8966266Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_cc3f7868-0b53-432e-9789-e9f0fb098405 2024-08-20T21:49:09.8967113Z NO_TD=False 2024-08-20T21:49:09.8967392Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T21:49:09.8967738Z _=/usr/bin/env 2024-08-20T21:49:09.8968050Z + echo 'Testing pytorch' 2024-08-20T21:49:09.8968373Z Testing pytorch 2024-08-20T21:49:09.8968677Z + export LANG=C.UTF-8 2024-08-20T21:49:09.8968984Z + LANG=C.UTF-8 2024-08-20T21:49:09.8969248Z + PR_NUMBER= 2024-08-20T21:49:09.8969704Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == \d\e\f\a\u\l\t ]] 2024-08-20T21:49:09.8970493Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T21:49:09.8971237Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == \s\l\o\w ]] 2024-08-20T21:49:09.8971955Z + [[ linux-jammy-py3.8-gcc11-build == *slow-gradcheck* ]] 2024-08-20T21:49:09.8972566Z + [[ linux-jammy-py3.8-gcc11-build == *cuda* ]] 2024-08-20T21:49:09.8973135Z + [[ linux-jammy-py3.8-gcc11-build == *rocm* ]] 2024-08-20T21:49:09.8973675Z + [[ linux-jammy-py3.8-gcc11-build == *xpu* ]] 2024-08-20T21:49:09.8974275Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *crossref* ]] 2024-08-20T21:49:09.8974939Z + [[ linux-jammy-py3.8-gcc11-build == *rocm* ]] 2024-08-20T21:49:09.8975587Z + [[ linux-jammy-py3.8-gcc11-build == *xpu* ]] 2024-08-20T21:49:09.8976148Z + [[ linux-jammy-py3.8-gcc11-build != *-bazel-* ]] 2024-08-20T21:49:09.8976678Z + pip_install --user ninja==1.10.2 2024-08-20T21:49:09.8977210Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T21:49:10.2982426Z Collecting ninja==1.10.2 2024-08-20T21:49:10.3093339Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T21:49:10.3203273Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T21:49:10.9582364Z Installing collected packages: ninja 2024-08-20T21:49:10.9665019Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T21:49:10.9666657Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T21:49:10.9717613Z Successfully installed ninja-1.10.2 2024-08-20T21:49:11.0330543Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:49:11.0332837Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:49:11.0334428Z + [[ linux-jammy-py3.8-gcc11-build == *aarch64* ]] 2024-08-20T21:49:11.0334901Z + install_tlparse 2024-08-20T21:49:11.0335265Z + pip_install --user tlparse==0.3.25 2024-08-20T21:49:11.0335830Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T21:49:11.4871518Z Collecting tlparse==0.3.25 2024-08-20T21:49:11.5051169Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T21:49:11.5695642Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T21:49:12.3444805Z Installing collected packages: tlparse 2024-08-20T21:49:12.4861558Z Successfully installed tlparse-0.3.25 2024-08-20T21:49:12.5559899Z ++ python -m site --user-base 2024-08-20T21:49:12.5824450Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:49:12.5826198Z + [[ linux-jammy-py3.8-gcc11-build == *asan* ]] 2024-08-20T21:49:12.5826783Z + [[ linux-jammy-py3.8-gcc11-build == *-debug* ]] 2024-08-20T21:49:12.5827381Z + [[ linux-jammy-py3.8-gcc11-build != *-bazel-* ]] 2024-08-20T21:49:12.5828304Z + echo 'We are not in debug mode: linux-jammy-py3.8-gcc11-build. Expect the assertion to pass' 2024-08-20T21:49:12.5829400Z We are not in debug mode: linux-jammy-py3.8-gcc11-build. Expect the assertion to pass 2024-08-20T21:49:12.5830061Z + cd test 2024-08-20T21:49:12.5830608Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T21:49:13.9463002Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T21:49:13.9464099Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T21:49:13.9464744Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T21:49:13.9465311Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *pr_time_benchmarks* ]] 2024-08-20T21:49:13.9466109Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *dynamo_eager* ]] 2024-08-20T21:49:13.9466852Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *aot_eager* ]] 2024-08-20T21:49:13.9467571Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *aot_inductor* ]] 2024-08-20T21:49:13.9468454Z + DYNAMO_BENCHMARK_FLAGS+=(--export-aot-inductor) 2024-08-20T21:49:13.9469074Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *dynamic* ]] 2024-08-20T21:49:13.9469843Z + DYNAMO_BENCHMARK_FLAGS+=(--dynamic-shapes --dynamic-batch-only) 2024-08-20T21:49:13.9470797Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *cpu* ]] 2024-08-20T21:49:13.9471425Z + DYNAMO_BENCHMARK_FLAGS+=(--device cpu) 2024-08-20T21:49:13.9486034Z + [[ linux-jammy-py3.8-gcc11-build == *libtorch* ]] 2024-08-20T21:49:13.9486676Z + [[ linux-jammy-py3.8-gcc11-build == *-bazel-* ]] 2024-08-20T21:49:13.9487982Z + cd test 2024-08-20T21:49:13.9488581Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T21:49:15.1080104Z PyTorch built with: 2024-08-20T21:49:15.1080780Z - GCC 11.4 2024-08-20T21:49:15.1081140Z - C++ Version: 201703 2024-08-20T21:49:15.1082118Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T21:49:15.1083682Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T21:49:15.1084377Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T21:49:15.1084885Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T21:49:15.1085381Z - NNPACK is enabled 2024-08-20T21:49:15.1085765Z - CPU capability usage: AVX512 2024-08-20T21:49:15.1093576Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T21:49:15.1100654Z 2024-08-20T21:49:15.2919639Z + cd test 2024-08-20T21:49:15.2920460Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T21:49:16.4349807Z ATen/Parallel: 2024-08-20T21:49:16.4350228Z at::get_num_threads() : 24 2024-08-20T21:49:16.4350619Z at::get_num_interop_threads() : 24 2024-08-20T21:49:16.4351039Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T21:49:16.4351465Z omp_get_max_threads() : 24 2024-08-20T21:49:16.4352571Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T21:49:16.4353412Z mkl_get_max_threads() : 24 2024-08-20T21:49:16.4354077Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T21:49:16.4354717Z std::thread::hardware_concurrency() : 48 2024-08-20T21:49:16.4355149Z Environment variables: 2024-08-20T21:49:16.4355492Z OMP_NUM_THREADS : [not set] 2024-08-20T21:49:16.4355858Z MKL_NUM_THREADS : [not set] 2024-08-20T21:49:16.4356216Z ATen parallel backend: OpenMP 2024-08-20T21:49:16.4356478Z 2024-08-20T21:49:16.6178902Z + [[ linux-jammy-py3.8-gcc11-build == *aarch64* ]] 2024-08-20T21:49:16.6179855Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *backward* ]] 2024-08-20T21:49:16.6180571Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *xla* ]] 2024-08-20T21:49:16.6181295Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *executorch* ]] 2024-08-20T21:49:16.6182103Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T21:49:16.6182901Z + [[ linux-jammy-py3.8-gcc11-build == *libtorch* ]] 2024-08-20T21:49:16.6183846Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == distributed ]] 2024-08-20T21:49:16.6184630Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *inductor_distributed* ]] 2024-08-20T21:49:16.6185552Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *inductor-halide* ]] 2024-08-20T21:49:16.6186484Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *inductor-micro-benchmark* ]] 2024-08-20T21:49:16.6187321Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *huggingface* ]] 2024-08-20T21:49:16.6188025Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *timm* ]] 2024-08-20T21:49:16.6188736Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *torchbench* ]] 2024-08-20T21:49:16.6189569Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *cpu* ]] 2024-08-20T21:49:16.6190112Z + install_torchaudio cpu 2024-08-20T21:49:16.6190433Z + local commit 2024-08-20T21:49:16.6190736Z ++ get_pinned_commit audio 2024-08-20T21:49:16.6191109Z ++ cat .github/ci_commit_pins/audio.txt 2024-08-20T21:49:16.6191610Z + commit=b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:16.6192060Z + [[ cpu == \c\u\d\a ]] 2024-08-20T21:49:16.6193000Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:16.6194570Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:16.9377379Z Collecting git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:16.9379112Z Cloning https://github.com/pytorch/audio.git (to revision b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862) to /tmp/pip-req-build-1nv6r4nk 2024-08-20T21:49:16.9411180Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/audio.git /tmp/pip-req-build-1nv6r4nk 2024-08-20T21:49:17.6465122Z Running command git rev-parse -q --verify 'sha^b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862' 2024-08-20T21:49:17.6501895Z Running command git fetch -q https://github.com/pytorch/audio.git b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:17.7542905Z Resolved https://github.com/pytorch/audio.git to commit b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T21:49:17.7544331Z Running command git submodule update --init --recursive -q 2024-08-20T21:49:19.6736421Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T21:49:19.6779757Z [?25hRequirement already satisfied: torch in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchaudio==2.4.0a0+b3f6f51) (2.5.0a0+git40ec5f6) 2024-08-20T21:49:19.6808302Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.13.1) 2024-08-20T21:49:19.6810218Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (4.12.2) 2024-08-20T21:49:19.6812057Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2.8.8) 2024-08-20T21:49:19.6814637Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (3.1.4) 2024-08-20T21:49:19.6817791Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (2024.6.1) 2024-08-20T21:49:19.6823932Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchaudio==2.4.0a0+b3f6f51) (1.12.1) 2024-08-20T21:49:19.6839216Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch->torchaudio==2.4.0a0+b3f6f51) (1.3.0) 2024-08-20T21:49:19.7202010Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch->torchaudio==2.4.0a0+b3f6f51) (2.1.5) 2024-08-20T21:49:19.7262156Z Building wheels for collected packages: torchaudio 2024-08-20T21:50:13.9916092Z Building wheel for torchaudio (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-08-20T21:50:13.9972313Z [?25h Created wheel for torchaudio: filename=torchaudio-2.4.0a0+b3f6f51-cp38-cp38-linux_x86_64.whl size=1767369 sha256=e3fde0581f17a86bed3b632460c4b8ba5aadafe267a75606eee33422be6ab0f1 2024-08-20T21:50:13.9974114Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/2d/7d/0d/bdfdb3f2b184f238c1d61b3f9b248992c036dfdb76e5397d45 2024-08-20T21:50:14.0009028Z Successfully built torchaudio 2024-08-20T21:50:14.5781097Z Installing collected packages: torchaudio 2024-08-20T21:50:14.7731186Z Successfully installed torchaudio-2.4.0a0+b3f6f51 2024-08-20T21:50:14.8854299Z + install_torchtext 2024-08-20T21:50:14.8855271Z + local data_commit 2024-08-20T21:50:14.8856111Z + local text_commit 2024-08-20T21:50:14.8858331Z ++ get_pinned_commit data 2024-08-20T21:50:14.8858711Z ++ cat .github/ci_commit_pins/data.txt 2024-08-20T21:50:14.8868215Z + data_commit=11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:14.8871189Z ++ get_pinned_commit text 2024-08-20T21:50:14.8871558Z ++ cat .github/ci_commit_pins/text.txt 2024-08-20T21:50:14.8881980Z + text_commit=b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:50:14.8883242Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:14.8884796Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:15.2064931Z Collecting git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:15.2069322Z Cloning https://github.com/pytorch/data.git (to revision 11bb5b847ea8b9e0d9bb82db3304daf383008d3f) to /tmp/pip-req-build-fv7d05fp 2024-08-20T21:50:15.2100735Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/data.git /tmp/pip-req-build-fv7d05fp 2024-08-20T21:50:15.6323808Z Running command git rev-parse -q --verify 'sha^11bb5b847ea8b9e0d9bb82db3304daf383008d3f' 2024-08-20T21:50:15.6358316Z Running command git fetch -q https://github.com/pytorch/data.git 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:15.7290874Z Running command git checkout -q 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:15.8687175Z Resolved https://github.com/pytorch/data.git to commit 11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T21:50:15.8688329Z Running command git submodule update --init --recursive -q 2024-08-20T21:51:04.8546025Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T21:51:04.8598656Z [?25hRequirement already satisfied: urllib3>=1.25 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchdata==0.7.0a0+11bb5b8) (1.26.19) 2024-08-20T21:51:04.8600395Z Requirement already satisfied: requests in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.32.3) 2024-08-20T21:51:04.8602245Z Requirement already satisfied: torch>2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchdata==0.7.0a0+11bb5b8) (2.5.0a0+git40ec5f6) 2024-08-20T21:51:04.8638069Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.13.1) 2024-08-20T21:51:04.8639976Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (4.12.2) 2024-08-20T21:51:04.8642016Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.8.8) 2024-08-20T21:51:04.8645964Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (3.1.4) 2024-08-20T21:51:04.8648921Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (2024.6.1) 2024-08-20T21:51:04.8654192Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.12.1) 2024-08-20T21:51:04.8666925Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch>2.0->torchdata==0.7.0a0+11bb5b8) (1.3.0) 2024-08-20T21:51:04.8728944Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.3.2) 2024-08-20T21:51:04.8731069Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (3.7) 2024-08-20T21:51:04.8733768Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchdata==0.7.0a0+11bb5b8) (2024.7.4) 2024-08-20T21:51:04.9120963Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch>2.0->torchdata==0.7.0a0+11bb5b8) (2.1.5) 2024-08-20T21:51:04.9183841Z Building wheels for collected packages: torchdata 2024-08-20T21:51:06.6808320Z Building wheel for torchdata (setup.py) ... [?25l- \ | done 2024-08-20T21:51:06.6820045Z [?25h Created wheel for torchdata: filename=torchdata-0.7.0a0+11bb5b8-py3-none-any.whl size=182970 sha256=93eaaf0d87e596e6e0528382409d76f7f2a3ca45f987b4e68c21f426ef8e29ec 2024-08-20T21:51:06.6821767Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/f2/b8/b4/f1929a760cf6ea9bda553072159f9dc096899c37ad8a5ffafe 2024-08-20T21:51:06.6859917Z Successfully built torchdata 2024-08-20T21:51:07.2458276Z Installing collected packages: torchdata 2024-08-20T21:51:07.3376244Z Successfully installed torchdata-0.7.0a0+11bb5b8 2024-08-20T21:51:09.3703706Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:09.3705336Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:09.6914860Z Collecting git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:09.6916895Z Cloning https://github.com/pytorch/text.git (to revision b0ebddc648d279826089db91775375221777a2db) to /tmp/pip-req-build-fre4mrf2 2024-08-20T21:51:09.6948629Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/text.git /tmp/pip-req-build-fre4mrf2 2024-08-20T21:51:10.6550067Z Running command git rev-parse -q --verify 'sha^b0ebddc648d279826089db91775375221777a2db' 2024-08-20T21:51:10.6585274Z Running command git fetch -q https://github.com/pytorch/text.git b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:10.7668740Z Running command git checkout -q b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:10.9248610Z Resolved https://github.com/pytorch/text.git to commit b0ebddc648d279826089db91775375221777a2db 2024-08-20T21:51:10.9249708Z Running command git submodule update --init --recursive -q 2024-08-20T21:51:17.4077071Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T21:51:17.4124435Z [?25hRequirement already satisfied: tqdm in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchtext==0.16.0a0+b0ebddc) (4.66.5) 2024-08-20T21:51:17.4126122Z Requirement already satisfied: requests in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.32.3) 2024-08-20T21:51:17.4127862Z Requirement already satisfied: torch in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchtext==0.16.0a0+b0ebddc) (2.5.0a0+git40ec5f6) 2024-08-20T21:51:17.4130776Z Requirement already satisfied: numpy in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchtext==0.16.0a0+b0ebddc) (1.21.2) 2024-08-20T21:51:17.4133973Z Requirement already satisfied: torchdata in /var/lib/jenkins/.local/lib/python3.8/site-packages (from torchtext==0.16.0a0+b0ebddc) (0.7.0a0+11bb5b8) 2024-08-20T21:51:17.4166844Z Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.3.2) 2024-08-20T21:51:17.4168720Z Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (3.7) 2024-08-20T21:51:17.4170609Z Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (1.26.19) 2024-08-20T21:51:17.4174601Z Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from requests->torchtext==0.16.0a0+b0ebddc) (2024.7.4) 2024-08-20T21:51:17.4205763Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.13.1) 2024-08-20T21:51:17.4207636Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (4.12.2) 2024-08-20T21:51:17.4209452Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2.8.8) 2024-08-20T21:51:17.4211522Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (3.1.4) 2024-08-20T21:51:17.4215379Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (2024.6.1) 2024-08-20T21:51:17.4221001Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchtext==0.16.0a0+b0ebddc) (1.12.1) 2024-08-20T21:51:17.4239079Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch->torchtext==0.16.0a0+b0ebddc) (1.3.0) 2024-08-20T21:51:17.4702292Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch->torchtext==0.16.0a0+b0ebddc) (2.1.5) 2024-08-20T21:51:17.4893380Z Building wheels for collected packages: torchtext 2024-08-20T21:51:45.7294419Z Building wheel for torchtext (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-08-20T21:51:45.7355754Z [?25h Created wheel for torchtext: filename=torchtext-0.16.0a0+b0ebddc-cp38-cp38-linux_x86_64.whl size=2045009 sha256=17dfe2db95c08f94792883bff50a85ffa7ff293406d937fae2f2ac7ffdd511ce 2024-08-20T21:51:45.7357515Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/d6/1b/67/0243915f173a4bc67968d5441b501e3df9c77a50521995f939 2024-08-20T21:51:45.7391315Z Successfully built torchtext 2024-08-20T21:51:46.2981381Z Installing collected packages: torchtext 2024-08-20T21:51:46.4137220Z Successfully installed torchtext-0.16.0a0+b0ebddc 2024-08-20T21:51:46.5193400Z + install_torchvision 2024-08-20T21:51:46.5193789Z + local orig_preload 2024-08-20T21:51:46.5194106Z + local commit 2024-08-20T21:51:46.5195983Z ++ get_pinned_commit vision 2024-08-20T21:51:46.5196432Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-20T21:51:46.5205535Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:46.5206058Z + orig_preload= 2024-08-20T21:51:46.5206526Z + '[' -n '' ']' 2024-08-20T21:51:46.5207405Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:46.5209212Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:46.8423557Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:46.8428107Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-gaa4s32g 2024-08-20T21:51:46.8461687Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-gaa4s32g 2024-08-20T21:51:48.1867586Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-20T21:51:48.1903940Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:48.3016919Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:48.5702368Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T21:51:50.5889990Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T21:51:50.5937529Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-08-20T21:51:50.5939430Z Requirement already satisfied: torch in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-20T21:51:50.5943238Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-20T21:51:50.6018915Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-20T21:51:50.6020836Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-20T21:51:50.6022693Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-20T21:51:50.6025887Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-20T21:51:50.6029016Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-20T21:51:50.6034543Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.12.1) 2024-08-20T21:51:50.6046861Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-20T21:51:50.6415598Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-20T21:51:50.6482716Z Building wheels for collected packages: torchvision 2024-08-20T21:52:16.4344175Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-08-20T21:52:16.4385471Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp38-cp38-linux_x86_64.whl size=1215626 sha256=b449ab9487b2402076e810f46a02ecef443c2e8b3934cecb43b7161838648629 2024-08-20T21:52:16.4387256Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/82/f9/1e/f126983e23cfd567a4cb268f7d0778919a2a7014fe9af04f11 2024-08-20T21:52:16.4561571Z Successfully built torchvision 2024-08-20T21:52:17.0270372Z Installing collected packages: torchvision 2024-08-20T21:52:17.4067437Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-20T21:52:17.4980911Z + '[' -n '' ']' 2024-08-20T21:52:17.4981334Z + TORCH_CUDA_ARCH_LIST='8.0;8.6' 2024-08-20T21:52:17.4981834Z + pip_install git+https://github.com/pytorch/ao.git 2024-08-20T21:52:17.4983618Z + pip install --progress-bar off git+https://github.com/pytorch/ao.git 2024-08-20T21:52:17.8201166Z Collecting git+https://github.com/pytorch/ao.git 2024-08-20T21:52:17.8202185Z Cloning https://github.com/pytorch/ao.git to /tmp/pip-req-build-ox50qdx_ 2024-08-20T21:52:17.8238571Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/ao.git /tmp/pip-req-build-ox50qdx_ 2024-08-20T21:52:18.3978593Z Resolved https://github.com/pytorch/ao.git to commit 2c8e3f368613f896fde14cc24b9268c9e125575e 2024-08-20T21:52:20.3229100Z Preparing metadata (setup.py) ... [?25l- done 2024-08-20T21:52:20.3281768Z [?25hBuilding wheels for collected packages: torchao 2024-08-20T21:52:34.8910925Z Building wheel for torchao (setup.py) ... [?25l- \ | / - \ | done 2024-08-20T21:52:34.9019323Z [?25h Created wheel for torchao: filename=torchao-0.4.0+git2c8e3f3-cp38-cp38-linux_x86_64.whl size=3735231 sha256=56aeb9a80d22288c406f9be783de276bcbdeb5c72bdab38c0e6d53c31f962466 2024-08-20T21:52:34.9021280Z Stored in directory: /tmp/pip-ephem-wheel-cache-orjeu0b_/wheels/7a/d0/ee/d845e81513be33ad662e10ba9fb85a811ff1ad92ea52f8b6a9 2024-08-20T21:52:34.9053719Z Successfully built torchao 2024-08-20T21:52:35.5289733Z Installing collected packages: torchao 2024-08-20T21:52:35.8024060Z Successfully installed torchao-0.4.0+git2c8e3f3 2024-08-20T21:52:35.8813190Z + id=0 2024-08-20T21:52:35.8813827Z + pip_install opencv-python==4.8.0.74 2024-08-20T21:52:35.8814477Z + pip install --progress-bar off opencv-python==4.8.0.74 2024-08-20T21:52:36.3200907Z Collecting opencv-python==4.8.0.74 2024-08-20T21:52:36.3332502Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB) 2024-08-20T21:52:36.3421452Z Requirement already satisfied: numpy>=1.17.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from opencv-python==4.8.0.74) (1.21.2) 2024-08-20T21:52:36.3500166Z Downloading opencv_python-4.8.0.74-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.7 MB) 2024-08-20T21:52:37.5375514Z Installing collected packages: opencv-python 2024-08-20T21:52:38.3945835Z Successfully installed opencv-python-4.8.0.74 2024-08-20T21:52:38.4708064Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *inductor_torchbench_smoketest_perf* ]] 2024-08-20T21:52:38.4709148Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *inductor_torchbench_cpu_smoketest_perf* ]] 2024-08-20T21:52:38.4710124Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *torchbench_gcp_smoketest* ]] 2024-08-20T21:52:38.4710788Z + checkout_install_torchbench 2024-08-20T21:52:38.4712720Z + local commit 2024-08-20T21:52:38.4713081Z ++ get_pinned_commit torchbench 2024-08-20T21:52:38.4713548Z ++ cat .github/ci_commit_pins/torchbench.txt 2024-08-20T21:52:38.4720362Z + commit=23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-20T21:52:38.4720979Z + git clone https://github.com/pytorch/benchmark torchbench 2024-08-20T21:52:38.4736553Z Cloning into 'torchbench'... 2024-08-20T21:52:38.6080803Z remote: Enumerating objects: 30997, done. 2024-08-20T21:52:38.6081540Z remote: Counting objects: 0% (1/1737) 2024-08-20T21:52:38.6082078Z remote: Counting objects: 1% (18/1737) 2024-08-20T21:52:38.6083070Z remote: Counting objects: 2% (35/1737) 2024-08-20T21:52:38.6083603Z remote: Counting objects: 3% (53/1737) 2024-08-20T21:52:38.6084140Z remote: Counting objects: 4% (70/1737) 2024-08-20T21:52:38.6084643Z remote: Counting objects: 5% (87/1737) 2024-08-20T21:52:38.6085169Z remote: Counting objects: 6% (105/1737) 2024-08-20T21:52:38.6085706Z remote: Counting objects: 7% (122/1737) 2024-08-20T21:52:38.6086240Z remote: Counting objects: 8% (139/1737) 2024-08-20T21:52:38.6086776Z remote: Counting objects: 9% (157/1737) 2024-08-20T21:52:38.6087302Z remote: Counting objects: 10% (174/1737) 2024-08-20T21:52:38.6087831Z remote: Counting objects: 11% (192/1737) 2024-08-20T21:52:38.6088353Z remote: Counting objects: 12% (209/1737) 2024-08-20T21:52:38.6089145Z remote: Counting objects: 13% (226/1737) 2024-08-20T21:52:38.6089686Z remote: Counting objects: 14% (244/1737) 2024-08-20T21:52:38.6090215Z remote: Counting objects: 15% (261/1737) 2024-08-20T21:52:38.6090727Z remote: Counting objects: 16% (278/1737) 2024-08-20T21:52:38.6091261Z remote: Counting objects: 17% (296/1737) 2024-08-20T21:52:38.6091783Z remote: Counting objects: 18% (313/1737) 2024-08-20T21:52:38.6092306Z remote: Counting objects: 19% (331/1737) 2024-08-20T21:52:38.6092810Z remote: Counting objects: 20% (348/1737) 2024-08-20T21:52:38.6093369Z remote: Counting objects: 21% (365/1737) 2024-08-20T21:52:38.6093897Z remote: Counting objects: 22% (383/1737) 2024-08-20T21:52:38.6094552Z remote: Counting objects: 23% (400/1737) 2024-08-20T21:52:38.6095075Z remote: Counting objects: 24% (417/1737) 2024-08-20T21:52:38.6095592Z remote: Counting objects: 25% (435/1737) 2024-08-20T21:52:38.6096116Z remote: Counting objects: 26% (452/1737) 2024-08-20T21:52:38.6096650Z remote: Counting objects: 27% (469/1737) 2024-08-20T21:52:38.6097172Z remote: Counting objects: 28% (487/1737) 2024-08-20T21:52:38.6097677Z remote: Counting objects: 29% (504/1737) 2024-08-20T21:52:38.6098320Z remote: Counting objects: 30% (522/1737) 2024-08-20T21:52:38.6098857Z remote: Counting objects: 31% (539/1737) 2024-08-20T21:52:38.6099382Z remote: Counting objects: 32% (556/1737) 2024-08-20T21:52:38.6099895Z remote: Counting objects: 33% (574/1737) 2024-08-20T21:52:38.6100419Z remote: Counting objects: 34% (591/1737) 2024-08-20T21:52:38.6100941Z remote: Counting objects: 35% (608/1737) 2024-08-20T21:52:38.6101455Z remote: Counting objects: 36% (626/1737) 2024-08-20T21:52:38.6101980Z remote: Counting objects: 37% (643/1737) 2024-08-20T21:52:38.6102507Z remote: Counting objects: 38% (661/1737) 2024-08-20T21:52:38.6103034Z remote: Counting objects: 39% (678/1737) 2024-08-20T21:52:38.6103548Z remote: Counting objects: 40% (695/1737) 2024-08-20T21:52:38.6104070Z remote: Counting objects: 41% (713/1737) 2024-08-20T21:52:38.6104598Z remote: Counting objects: 42% (730/1737) 2024-08-20T21:52:38.6105121Z remote: Counting objects: 43% (747/1737) 2024-08-20T21:52:38.6105625Z remote: Counting objects: 44% (765/1737) 2024-08-20T21:52:38.6106155Z remote: Counting objects: 45% (782/1737) 2024-08-20T21:52:38.6106677Z remote: Counting objects: 46% (800/1737) 2024-08-20T21:52:38.6107199Z remote: Counting objects: 47% (817/1737) 2024-08-20T21:52:38.6107712Z remote: Counting objects: 48% (834/1737) 2024-08-20T21:52:38.6108238Z remote: Counting objects: 49% (852/1737) 2024-08-20T21:52:38.6108768Z remote: Counting objects: 50% (869/1737) 2024-08-20T21:52:38.6109292Z remote: Counting objects: 51% (886/1737) 2024-08-20T21:52:38.6109800Z remote: Counting objects: 52% (904/1737) 2024-08-20T21:52:38.6110321Z remote: Counting objects: 53% (921/1737) 2024-08-20T21:52:38.6110850Z remote: Counting objects: 54% (938/1737) 2024-08-20T21:52:38.6111356Z remote: Counting objects: 55% (956/1737) 2024-08-20T21:52:38.6111874Z remote: Counting objects: 56% (973/1737) 2024-08-20T21:52:38.6112401Z remote: Counting objects: 57% (991/1737) 2024-08-20T21:52:38.6112932Z remote: Counting objects: 58% (1008/1737) 2024-08-20T21:52:38.6113453Z remote: Counting objects: 59% (1025/1737) 2024-08-20T21:52:38.6113989Z remote: Counting objects: 60% (1043/1737) 2024-08-20T21:52:38.6114527Z remote: Counting objects: 61% (1060/1737) 2024-08-20T21:52:38.6115059Z remote: Counting objects: 62% (1077/1737) 2024-08-20T21:52:38.6115584Z remote: Counting objects: 63% (1095/1737) 2024-08-20T21:52:38.6116121Z remote: Counting objects: 64% (1112/1737) 2024-08-20T21:52:38.6116660Z remote: Counting objects: 65% (1130/1737) 2024-08-20T21:52:38.6117190Z remote: Counting objects: 66% (1147/1737) 2024-08-20T21:52:38.6117704Z remote: Counting objects: 67% (1164/1737) 2024-08-20T21:52:38.6118338Z remote: Counting objects: 68% (1182/1737) 2024-08-20T21:52:38.6118872Z remote: Counting objects: 69% (1199/1737) 2024-08-20T21:52:38.6119403Z remote: Counting objects: 70% (1216/1737) 2024-08-20T21:52:38.6119921Z remote: Counting objects: 71% (1234/1737) 2024-08-20T21:52:38.6120451Z remote: Counting objects: 72% (1251/1737) 2024-08-20T21:52:38.6120982Z remote: Counting objects: 73% (1269/1737) 2024-08-20T21:52:38.6121528Z remote: Counting objects: 74% (1286/1737) 2024-08-20T21:52:38.6122036Z remote: Counting objects: 75% (1303/1737) 2024-08-20T21:52:38.6122557Z remote: Counting objects: 76% (1321/1737) 2024-08-20T21:52:38.6123082Z remote: Counting objects: 77% (1338/1737) 2024-08-20T21:52:38.6123667Z remote: Counting objects: 78% (1355/1737) 2024-08-20T21:52:38.6124189Z remote: Counting objects: 79% (1373/1737) 2024-08-20T21:52:38.6124717Z remote: Counting objects: 80% (1390/1737) 2024-08-20T21:52:38.6125247Z remote: Counting objects: 81% (1407/1737) 2024-08-20T21:52:38.6125759Z remote: Counting objects: 82% (1425/1737) 2024-08-20T21:52:38.6126291Z remote: Counting objects: 83% (1442/1737) 2024-08-20T21:52:38.6126815Z remote: Counting objects: 84% (1460/1737) 2024-08-20T21:52:38.6127337Z remote: Counting objects: 85% (1477/1737) 2024-08-20T21:52:38.6127847Z remote: Counting objects: 86% (1494/1737) 2024-08-20T21:52:38.6128372Z remote: Counting objects: 87% (1512/1737) 2024-08-20T21:52:38.6128894Z remote: Counting objects: 88% (1529/1737) 2024-08-20T21:52:38.6129416Z remote: Counting objects: 89% (1546/1737) 2024-08-20T21:52:38.6129923Z remote: Counting objects: 90% (1564/1737) 2024-08-20T21:52:38.6130452Z remote: Counting objects: 91% (1581/1737) 2024-08-20T21:52:38.6130970Z remote: Counting objects: 92% (1599/1737) 2024-08-20T21:52:38.6131494Z remote: Counting objects: 93% (1616/1737) 2024-08-20T21:52:38.6132003Z remote: Counting objects: 94% (1633/1737) 2024-08-20T21:52:38.6132529Z remote: Counting objects: 95% (1651/1737) 2024-08-20T21:52:38.6133049Z remote: Counting objects: 96% (1668/1737) 2024-08-20T21:52:38.6133572Z remote: Counting objects: 97% (1685/1737) 2024-08-20T21:52:38.6134081Z remote: Counting objects: 98% (1703/1737) 2024-08-20T21:52:38.6134605Z remote: Counting objects: 99% (1720/1737) 2024-08-20T21:52:38.6135128Z remote: Counting objects: 100% (1737/1737) 2024-08-20T21:52:38.6135680Z remote: Counting objects: 100% (1737/1737), done. 2024-08-20T21:52:38.6136257Z remote: Compressing objects: 0% (1/895) 2024-08-20T21:52:38.6201613Z remote: Compressing objects: 1% (9/895) 2024-08-20T21:52:38.6319381Z remote: Compressing objects: 2% (18/895) 2024-08-20T21:52:38.6399281Z remote: Compressing objects: 3% (27/895) 2024-08-20T21:52:38.6446159Z remote: Compressing objects: 4% (36/895) 2024-08-20T21:52:38.6500955Z remote: Compressing objects: 5% (45/895) 2024-08-20T21:52:38.6531819Z remote: Compressing objects: 6% (54/895) 2024-08-20T21:52:38.6560813Z remote: Compressing objects: 7% (63/895) 2024-08-20T21:52:38.6595060Z remote: Compressing objects: 8% (72/895) 2024-08-20T21:52:38.6628762Z remote: Compressing objects: 9% (81/895) 2024-08-20T21:52:38.6659552Z remote: Compressing objects: 10% (90/895) 2024-08-20T21:52:38.6671658Z remote: Compressing objects: 11% (99/895) 2024-08-20T21:52:38.6680762Z remote: Compressing objects: 12% (108/895) 2024-08-20T21:52:38.6700945Z remote: Compressing objects: 13% (117/895) 2024-08-20T21:52:38.6717949Z remote: Compressing objects: 14% (126/895) 2024-08-20T21:52:38.6724332Z remote: Compressing objects: 15% (135/895) 2024-08-20T21:52:38.6733824Z remote: Compressing objects: 16% (144/895) 2024-08-20T21:52:38.6739066Z remote: Compressing objects: 17% (153/895) 2024-08-20T21:52:38.6740685Z remote: Compressing objects: 18% (162/895) 2024-08-20T21:52:38.6741808Z remote: Compressing objects: 19% (171/895) 2024-08-20T21:52:38.6742605Z remote: Compressing objects: 20% (179/895) 2024-08-20T21:52:38.6743150Z remote: Compressing objects: 21% (188/895) 2024-08-20T21:52:38.6743867Z remote: Compressing objects: 22% (197/895) 2024-08-20T21:52:38.6745018Z remote: Compressing objects: 23% (206/895) 2024-08-20T21:52:38.6746523Z remote: Compressing objects: 24% (215/895) 2024-08-20T21:52:38.6761245Z remote: Compressing objects: 25% (224/895) 2024-08-20T21:52:38.6761941Z remote: Compressing objects: 26% (233/895) 2024-08-20T21:52:38.6767924Z remote: Compressing objects: 27% (242/895) 2024-08-20T21:52:38.6768484Z remote: Compressing objects: 28% (251/895) 2024-08-20T21:52:38.6775455Z remote: Compressing objects: 29% (260/895) 2024-08-20T21:52:38.6776202Z remote: Compressing objects: 30% (269/895) 2024-08-20T21:52:38.6782556Z remote: Compressing objects: 31% (278/895) 2024-08-20T21:52:38.6784151Z remote: Compressing objects: 32% (287/895) 2024-08-20T21:52:38.6788581Z remote: Compressing objects: 33% (296/895) 2024-08-20T21:52:38.6790153Z remote: Compressing objects: 34% (305/895) 2024-08-20T21:52:38.6793525Z remote: Compressing objects: 35% (314/895) 2024-08-20T21:52:38.6794080Z remote: Compressing objects: 36% (323/895) 2024-08-20T21:52:38.6794637Z remote: Compressing objects: 37% (332/895) 2024-08-20T21:52:38.6795171Z remote: Compressing objects: 38% (341/895) 2024-08-20T21:52:38.6798279Z remote: Compressing objects: 39% (350/895) 2024-08-20T21:52:38.6798830Z remote: Compressing objects: 40% (358/895) 2024-08-20T21:52:38.6804308Z remote: Compressing objects: 41% (367/895) 2024-08-20T21:52:38.6804840Z remote: Compressing objects: 42% (376/895) 2024-08-20T21:52:38.6805410Z remote: Compressing objects: 43% (385/895) 2024-08-20T21:52:38.6805963Z remote: Compressing objects: 44% (394/895) 2024-08-20T21:52:38.6806513Z remote: Compressing objects: 45% (403/895) 2024-08-20T21:52:38.6807043Z remote: Compressing objects: 46% (412/895) 2024-08-20T21:52:38.6807601Z remote: Compressing objects: 47% (421/895) 2024-08-20T21:52:38.6808144Z remote: Compressing objects: 48% (430/895) 2024-08-20T21:52:38.6808689Z remote: Compressing objects: 49% (439/895) 2024-08-20T21:52:38.6809220Z remote: Compressing objects: 50% (448/895) 2024-08-20T21:52:38.6809768Z remote: Compressing objects: 51% (457/895) 2024-08-20T21:52:38.6810315Z remote: Compressing objects: 52% (466/895) 2024-08-20T21:52:38.6810858Z remote: Compressing objects: 53% (475/895) 2024-08-20T21:52:38.6811384Z remote: Compressing objects: 54% (484/895) 2024-08-20T21:52:38.6811928Z remote: Compressing objects: 55% (493/895) 2024-08-20T21:52:38.6812469Z remote: Compressing objects: 56% (502/895) 2024-08-20T21:52:38.6813015Z remote: Compressing objects: 57% (511/895) 2024-08-20T21:52:38.6813558Z remote: Compressing objects: 58% (520/895) 2024-08-20T21:52:38.6814101Z remote: Compressing objects: 59% (529/895) 2024-08-20T21:52:38.6814644Z remote: Compressing objects: 60% (537/895) 2024-08-20T21:52:38.6815195Z remote: Compressing objects: 61% (546/895) 2024-08-20T21:52:38.6815723Z remote: Compressing objects: 62% (555/895) 2024-08-20T21:52:38.6816270Z remote: Compressing objects: 63% (564/895) 2024-08-20T21:52:38.6816823Z remote: Compressing objects: 64% (573/895) 2024-08-20T21:52:38.6817366Z remote: Compressing objects: 65% (582/895) 2024-08-20T21:52:38.6817897Z remote: Compressing objects: 66% (591/895) 2024-08-20T21:52:38.6818569Z remote: Compressing objects: 67% (600/895) 2024-08-20T21:52:38.6819134Z remote: Compressing objects: 68% (609/895) 2024-08-20T21:52:38.6819680Z remote: Compressing objects: 69% (618/895) 2024-08-20T21:52:38.6820217Z remote: Compressing objects: 70% (627/895) 2024-08-20T21:52:38.6820769Z remote: Compressing objects: 71% (636/895) 2024-08-20T21:52:38.6821313Z remote: Compressing objects: 72% (645/895) 2024-08-20T21:52:38.6821857Z remote: Compressing objects: 73% (654/895) 2024-08-20T21:52:38.6822534Z remote: Compressing objects: 74% (663/895) 2024-08-20T21:52:38.6823095Z remote: Compressing objects: 75% (672/895) 2024-08-20T21:52:38.6823646Z remote: Compressing objects: 76% (681/895) 2024-08-20T21:52:38.6827466Z remote: Compressing objects: 77% (690/895) 2024-08-20T21:52:38.6832528Z remote: Compressing objects: 78% (699/895) 2024-08-20T21:52:38.6838999Z remote: Compressing objects: 79% (708/895) 2024-08-20T21:52:38.6844090Z remote: Compressing objects: 80% (716/895) 2024-08-20T21:52:38.6861736Z remote: Compressing objects: 81% (725/895) 2024-08-20T21:52:38.6862328Z remote: Compressing objects: 82% (734/895) 2024-08-20T21:52:38.6862890Z remote: Compressing objects: 83% (743/895) 2024-08-20T21:52:38.6865934Z remote: Compressing objects: 84% (752/895) 2024-08-20T21:52:38.6870501Z remote: Compressing objects: 85% (761/895) 2024-08-20T21:52:38.6875419Z remote: Compressing objects: 86% (770/895) 2024-08-20T21:52:38.6881071Z remote: Compressing objects: 87% (779/895) 2024-08-20T21:52:38.6885450Z remote: Compressing objects: 88% (788/895) 2024-08-20T21:52:38.6888270Z remote: Compressing objects: 89% (797/895) 2024-08-20T21:52:38.6891594Z remote: Compressing objects: 90% (806/895) 2024-08-20T21:52:38.6893683Z remote: Compressing objects: 91% (815/895) 2024-08-20T21:52:38.6897029Z remote: Compressing objects: 92% (824/895) 2024-08-20T21:52:38.6900288Z remote: Compressing objects: 93% (833/895) 2024-08-20T21:52:38.6902291Z remote: Compressing objects: 94% (842/895) 2024-08-20T21:52:38.6904994Z remote: Compressing objects: 95% (851/895) 2024-08-20T21:52:38.6908155Z remote: Compressing objects: 96% (860/895) 2024-08-20T21:52:38.6910419Z remote: Compressing objects: 97% (869/895) 2024-08-20T21:52:38.6911826Z remote: Compressing objects: 98% (878/895) 2024-08-20T21:52:38.6913647Z remote: Compressing objects: 99% (887/895) 2024-08-20T21:52:38.6914178Z remote: Compressing objects: 100% (895/895) 2024-08-20T21:52:38.6914774Z remote: Compressing objects: 100% (895/895), done. 2024-08-20T21:52:38.6971359Z Receiving objects: 0% (1/30997) 2024-08-20T21:52:38.7009261Z Receiving objects: 1% (310/30997) 2024-08-20T21:52:38.7044392Z Receiving objects: 2% (620/30997) 2024-08-20T21:52:38.7079177Z Receiving objects: 3% (930/30997) 2024-08-20T21:52:38.7105414Z Receiving objects: 4% (1240/30997) 2024-08-20T21:52:38.7139564Z Receiving objects: 5% (1550/30997) 2024-08-20T21:52:38.7175350Z Receiving objects: 6% (1860/30997) 2024-08-20T21:52:38.7211196Z Receiving objects: 7% (2170/30997) 2024-08-20T21:52:38.7238851Z Receiving objects: 8% (2480/30997) 2024-08-20T21:52:38.7265220Z Receiving objects: 9% (2790/30997) 2024-08-20T21:52:38.7333504Z Receiving objects: 10% (3100/30997) 2024-08-20T21:52:38.7379620Z Receiving objects: 11% (3410/30997) 2024-08-20T21:52:38.7441908Z Receiving objects: 12% (3720/30997) 2024-08-20T21:52:38.7567178Z Receiving objects: 13% (4030/30997) 2024-08-20T21:52:38.7697582Z Receiving objects: 14% (4340/30997) 2024-08-20T21:52:38.7762160Z Receiving objects: 15% (4650/30997) 2024-08-20T21:52:38.7829512Z Receiving objects: 16% (4960/30997) 2024-08-20T21:52:38.7884586Z Receiving objects: 17% (5270/30997) 2024-08-20T21:52:38.7912023Z Receiving objects: 18% (5580/30997) 2024-08-20T21:52:38.7967913Z Receiving objects: 19% (5890/30997) 2024-08-20T21:52:38.7999954Z Receiving objects: 20% (6200/30997) 2024-08-20T21:52:38.8013810Z Receiving objects: 21% (6510/30997) 2024-08-20T21:52:38.8052542Z Receiving objects: 22% (6820/30997) 2024-08-20T21:52:39.6257120Z Receiving objects: 23% (7130/30997) 2024-08-20T21:52:39.6927642Z Receiving objects: 24% (7440/30997), 43.54 MiB | 87.05 MiB/s 2024-08-20T21:52:40.1058650Z Receiving objects: 24% (7534/30997), 92.08 MiB | 92.07 MiB/s 2024-08-20T21:52:40.1660360Z Receiving objects: 25% (7750/30997), 92.08 MiB | 92.07 MiB/s 2024-08-20T21:52:40.2259977Z Receiving objects: 26% (8060/30997), 92.08 MiB | 92.07 MiB/s 2024-08-20T21:52:40.2857414Z Receiving objects: 27% (8370/30997), 132.72 MiB | 88.48 MiB/s 2024-08-20T21:52:40.3449672Z Receiving objects: 28% (8680/30997), 132.72 MiB | 88.48 MiB/s 2024-08-20T21:52:40.4050710Z Receiving objects: 29% (8990/30997), 132.72 MiB | 88.48 MiB/s 2024-08-20T21:52:40.4657406Z Receiving objects: 30% (9300/30997), 132.72 MiB | 88.48 MiB/s 2024-08-20T21:52:40.5255830Z Receiving objects: 31% (9610/30997), 132.72 MiB | 88.48 MiB/s 2024-08-20T21:52:40.6929284Z Receiving objects: 32% (9920/30997), 132.72 MiB | 88.48 MiB/s 2024-08-20T21:52:40.7281808Z Receiving objects: 32% (10189/30997), 178.77 MiB | 89.38 MiB/s 2024-08-20T21:52:40.7972474Z Receiving objects: 33% (10230/30997), 178.77 MiB | 89.38 MiB/s 2024-08-20T21:52:40.8088106Z Receiving objects: 34% (10539/30997), 178.77 MiB | 89.38 MiB/s 2024-08-20T21:52:41.0397086Z Receiving objects: 35% (10849/30997), 178.77 MiB | 89.38 MiB/s 2024-08-20T21:52:41.2461071Z Receiving objects: 36% (11159/30997), 178.77 MiB | 89.38 MiB/s 2024-08-20T21:52:41.2479250Z Receiving objects: 37% (11469/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2499805Z Receiving objects: 38% (11779/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2517102Z Receiving objects: 39% (12089/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2531230Z Receiving objects: 40% (12399/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2535687Z Receiving objects: 41% (12709/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2547140Z Receiving objects: 42% (13019/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2566006Z Receiving objects: 43% (13329/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2578796Z Receiving objects: 44% (13639/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2582998Z Receiving objects: 45% (13949/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2586982Z Receiving objects: 46% (14259/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2605305Z Receiving objects: 47% (14569/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2620439Z Receiving objects: 48% (14879/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2637279Z Receiving objects: 49% (15189/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2820117Z Receiving objects: 50% (15499/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2822865Z Receiving objects: 51% (15809/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2828769Z Receiving objects: 52% (16119/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2834298Z Receiving objects: 53% (16429/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2838871Z Receiving objects: 54% (16739/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2854720Z Receiving objects: 55% (17049/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2864181Z Receiving objects: 56% (17359/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2869604Z Receiving objects: 57% (17669/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2886789Z Receiving objects: 58% (17979/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2893121Z Receiving objects: 59% (18289/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.2902232Z Receiving objects: 60% (18599/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3019845Z Receiving objects: 61% (18909/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3021251Z Receiving objects: 62% (19219/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3026035Z Receiving objects: 63% (19529/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3033171Z Receiving objects: 64% (19839/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3039791Z Receiving objects: 65% (20149/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3047667Z Receiving objects: 66% (20459/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3264891Z Receiving objects: 67% (20768/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3640327Z Receiving objects: 68% (21078/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3642169Z Receiving objects: 69% (21388/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3654988Z Receiving objects: 70% (21698/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3664513Z Receiving objects: 71% (22008/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3682109Z Receiving objects: 72% (22318/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3685799Z Receiving objects: 73% (22628/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3699113Z Receiving objects: 74% (22938/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3709179Z Receiving objects: 75% (23248/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3715321Z Receiving objects: 76% (23558/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3722274Z Receiving objects: 77% (23868/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3930463Z Receiving objects: 78% (24178/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3961973Z Receiving objects: 79% (24488/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3965627Z Receiving objects: 80% (24798/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3976017Z Receiving objects: 81% (25108/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.3985878Z Receiving objects: 82% (25418/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4035894Z Receiving objects: 83% (25728/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4147041Z Receiving objects: 84% (26038/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4151026Z Receiving objects: 85% (26348/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4157743Z Receiving objects: 86% (26658/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4162340Z Receiving objects: 87% (26968/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4168846Z Receiving objects: 88% (27278/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4173528Z Receiving objects: 89% (27588/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4283954Z Receiving objects: 90% (27898/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4297977Z Receiving objects: 91% (28208/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4303414Z Receiving objects: 92% (28518/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4323245Z Receiving objects: 93% (28828/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4349820Z Receiving objects: 94% (29138/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4360060Z Receiving objects: 95% (29448/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4364767Z Receiving objects: 96% (29758/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4371042Z Receiving objects: 97% (30068/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4444199Z Receiving objects: 98% (30378/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4460515Z Receiving objects: 99% (30688/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4461570Z remote: Total 30997 (delta 1014), reused 1470 (delta 823), pack-reused 29260 (from 1) 2024-08-20T21:52:41.4479298Z Receiving objects: 100% (30997/30997), 216.55 MiB | 86.62 MiB/s 2024-08-20T21:52:41.4480042Z Receiving objects: 100% (30997/30997), 232.50 MiB | 84.39 MiB/s, done. 2024-08-20T21:52:41.4510342Z Resolving deltas: 0% (0/16727) 2024-08-20T21:52:41.4514226Z Resolving deltas: 1% (168/16727) 2024-08-20T21:52:41.4516520Z Resolving deltas: 2% (336/16727) 2024-08-20T21:52:41.4521264Z Resolving deltas: 3% (502/16727) 2024-08-20T21:52:41.4525454Z Resolving deltas: 4% (670/16727) 2024-08-20T21:52:41.4527830Z Resolving deltas: 5% (838/16727) 2024-08-20T21:52:41.4529928Z Resolving deltas: 6% (1004/16727) 2024-08-20T21:52:41.4533891Z Resolving deltas: 7% (1171/16727) 2024-08-20T21:52:41.4537438Z Resolving deltas: 8% (1339/16727) 2024-08-20T21:52:41.4541093Z Resolving deltas: 9% (1506/16727) 2024-08-20T21:52:41.4544468Z Resolving deltas: 10% (1673/16727) 2024-08-20T21:52:41.4548403Z Resolving deltas: 11% (1840/16727) 2024-08-20T21:52:41.4551297Z Resolving deltas: 12% (2008/16727) 2024-08-20T21:52:41.4555211Z Resolving deltas: 13% (2175/16727) 2024-08-20T21:52:41.4559082Z Resolving deltas: 14% (2342/16727) 2024-08-20T21:52:41.4563054Z Resolving deltas: 15% (2511/16727) 2024-08-20T21:52:41.4566998Z Resolving deltas: 16% (2677/16727) 2024-08-20T21:52:41.4571773Z Resolving deltas: 17% (2844/16727) 2024-08-20T21:52:41.4576813Z Resolving deltas: 18% (3011/16727) 2024-08-20T21:52:41.4581427Z Resolving deltas: 19% (3179/16727) 2024-08-20T21:52:41.4584467Z Resolving deltas: 20% (3346/16727) 2024-08-20T21:52:41.4588539Z Resolving deltas: 21% (3514/16727) 2024-08-20T21:52:41.4594477Z Resolving deltas: 22% (3680/16727) 2024-08-20T21:52:41.4600748Z Resolving deltas: 23% (3848/16727) 2024-08-20T21:52:41.4608685Z Resolving deltas: 24% (4015/16727) 2024-08-20T21:52:41.4615444Z Resolving deltas: 25% (4182/16727) 2024-08-20T21:52:41.4619961Z Resolving deltas: 26% (4350/16727) 2024-08-20T21:52:41.4627784Z Resolving deltas: 27% (4517/16727) 2024-08-20T21:52:41.4632046Z Resolving deltas: 28% (4684/16727) 2024-08-20T21:52:41.4638073Z Resolving deltas: 29% (4852/16727) 2024-08-20T21:52:41.4647977Z Resolving deltas: 30% (5019/16727) 2024-08-20T21:52:41.4651363Z Resolving deltas: 31% (5186/16727) 2024-08-20T21:52:41.4654280Z Resolving deltas: 32% (5353/16727) 2024-08-20T21:52:41.4657075Z Resolving deltas: 33% (5520/16727) 2024-08-20T21:52:41.4659689Z Resolving deltas: 34% (5688/16727) 2024-08-20T21:52:41.4662259Z Resolving deltas: 35% (5855/16727) 2024-08-20T21:52:41.4665052Z Resolving deltas: 36% (6022/16727) 2024-08-20T21:52:41.4667864Z Resolving deltas: 37% (6189/16727) 2024-08-20T21:52:41.4671778Z Resolving deltas: 38% (6357/16727) 2024-08-20T21:52:41.4675238Z Resolving deltas: 39% (6524/16727) 2024-08-20T21:52:41.4680402Z Resolving deltas: 40% (6691/16727) 2024-08-20T21:52:41.4687937Z Resolving deltas: 41% (6859/16727) 2024-08-20T21:52:41.4694836Z Resolving deltas: 42% (7026/16727) 2024-08-20T21:52:41.4702225Z Resolving deltas: 43% (7193/16727) 2024-08-20T21:52:41.4707111Z Resolving deltas: 44% (7360/16727) 2024-08-20T21:52:41.4710048Z Resolving deltas: 45% (7528/16727) 2024-08-20T21:52:41.4712779Z Resolving deltas: 46% (7695/16727) 2024-08-20T21:52:41.4715788Z Resolving deltas: 47% (7862/16727) 2024-08-20T21:52:41.4720375Z Resolving deltas: 48% (8030/16727) 2024-08-20T21:52:41.4738912Z Resolving deltas: 49% (8198/16727) 2024-08-20T21:52:41.4741873Z Resolving deltas: 50% (8364/16727) 2024-08-20T21:52:41.4747042Z Resolving deltas: 51% (8531/16727) 2024-08-20T21:52:41.4752795Z Resolving deltas: 52% (8699/16727) 2024-08-20T21:52:41.4762085Z Resolving deltas: 53% (8866/16727) 2024-08-20T21:52:41.4766155Z Resolving deltas: 54% (9033/16727) 2024-08-20T21:52:41.4769909Z Resolving deltas: 55% (9201/16727) 2024-08-20T21:52:41.4773486Z Resolving deltas: 56% (9368/16727) 2024-08-20T21:52:41.4777340Z Resolving deltas: 57% (9535/16727) 2024-08-20T21:52:41.4782458Z Resolving deltas: 58% (9702/16727) 2024-08-20T21:52:41.4788979Z Resolving deltas: 59% (9869/16727) 2024-08-20T21:52:41.4793551Z Resolving deltas: 60% (10037/16727) 2024-08-20T21:52:41.4800022Z Resolving deltas: 61% (10204/16727) 2024-08-20T21:52:41.4806180Z Resolving deltas: 62% (10371/16727) 2024-08-20T21:52:41.4811972Z Resolving deltas: 63% (10539/16727) 2024-08-20T21:52:41.4820307Z Resolving deltas: 64% (10706/16727) 2024-08-20T21:52:41.4827229Z Resolving deltas: 65% (10873/16727) 2024-08-20T21:52:41.4834036Z Resolving deltas: 66% (11040/16727) 2024-08-20T21:52:41.4839215Z Resolving deltas: 67% (11209/16727) 2024-08-20T21:52:41.4842123Z Resolving deltas: 68% (11376/16727) 2024-08-20T21:52:41.4847855Z Resolving deltas: 69% (11542/16727) 2024-08-20T21:52:41.4852998Z Resolving deltas: 70% (11709/16727) 2024-08-20T21:52:41.4857589Z Resolving deltas: 71% (11877/16727) 2024-08-20T21:52:41.4861595Z Resolving deltas: 72% (12045/16727) 2024-08-20T21:52:41.4867927Z Resolving deltas: 73% (12212/16727) 2024-08-20T21:52:41.4875646Z Resolving deltas: 74% (12378/16727) 2024-08-20T21:52:41.4881005Z Resolving deltas: 75% (12547/16727) 2024-08-20T21:52:41.4886279Z Resolving deltas: 76% (12713/16727) 2024-08-20T21:52:41.4890776Z Resolving deltas: 77% (12880/16727) 2024-08-20T21:52:41.4897416Z Resolving deltas: 78% (13050/16727) 2024-08-20T21:52:41.4903808Z Resolving deltas: 79% (13215/16727) 2024-08-20T21:52:41.4910352Z Resolving deltas: 80% (13382/16727) 2024-08-20T21:52:41.4914971Z Resolving deltas: 81% (13549/16727) 2024-08-20T21:52:41.4921727Z Resolving deltas: 82% (13717/16727) 2024-08-20T21:52:41.4928976Z Resolving deltas: 83% (13884/16727) 2024-08-20T21:52:41.4935167Z Resolving deltas: 84% (14051/16727) 2024-08-20T21:52:41.4939681Z Resolving deltas: 85% (14218/16727) 2024-08-20T21:52:41.4944069Z Resolving deltas: 86% (14386/16727) 2024-08-20T21:52:41.4947461Z Resolving deltas: 87% (14553/16727) 2024-08-20T21:52:41.4951570Z Resolving deltas: 88% (14720/16727) 2024-08-20T21:52:41.4956783Z Resolving deltas: 89% (14888/16727) 2024-08-20T21:52:41.4962582Z Resolving deltas: 90% (15055/16727) 2024-08-20T21:52:41.4967064Z Resolving deltas: 91% (15222/16727) 2024-08-20T21:52:41.4970859Z Resolving deltas: 92% (15389/16727) 2024-08-20T21:52:41.4974776Z Resolving deltas: 93% (15557/16727) 2024-08-20T21:52:41.4981116Z Resolving deltas: 94% (15725/16727) 2024-08-20T21:52:41.4986912Z Resolving deltas: 95% (15891/16727) 2024-08-20T21:52:41.4993291Z Resolving deltas: 96% (16058/16727) 2024-08-20T21:52:41.4999015Z Resolving deltas: 97% (16226/16727) 2024-08-20T21:52:41.5004094Z Resolving deltas: 98% (16393/16727) 2024-08-20T21:52:41.5135503Z Resolving deltas: 99% (16560/16727) 2024-08-20T21:52:41.5135927Z Resolving deltas: 100% (16727/16727) 2024-08-20T21:52:41.5136358Z Resolving deltas: 100% (16727/16727), done. 2024-08-20T21:52:42.4762991Z + pushd torchbench 2024-08-20T21:52:42.4763459Z ~/workspace/torchbench ~/workspace 2024-08-20T21:52:42.4763960Z + git checkout 23512dbebd44a11eb84afbf53c3c071dd105297e 2024-08-20T21:52:42.5366869Z Note: switching to '23512dbebd44a11eb84afbf53c3c071dd105297e'. 2024-08-20T21:52:42.5367360Z 2024-08-20T21:52:42.5367758Z You are in 'detached HEAD' state. You can look around, make experimental 2024-08-20T21:52:42.5368572Z changes and commit them, and you can discard any commits you make in this 2024-08-20T21:52:42.5369355Z state without impacting any branches by switching back to a branch. 2024-08-20T21:52:42.5369823Z 2024-08-20T21:52:42.5370131Z If you want to create a new branch to retain commits you create, you may 2024-08-20T21:52:42.5370947Z do so (now or later) by using -c with the switch command. Example: 2024-08-20T21:52:42.5371379Z 2024-08-20T21:52:42.5371572Z git switch -c 2024-08-20T21:52:42.5371839Z 2024-08-20T21:52:42.5371975Z Or undo this operation with: 2024-08-20T21:52:42.5372214Z 2024-08-20T21:52:42.5372318Z git switch - 2024-08-20T21:52:42.5372502Z 2024-08-20T21:52:42.5372839Z Turn off this advice by setting config variable advice.detachedHead to false 2024-08-20T21:52:42.5373367Z 2024-08-20T21:52:42.5373519Z HEAD is now at 23512dbe Pin numpy version 2024-08-20T21:52:42.5373963Z + '[' '' ']' 2024-08-20T21:52:42.5374307Z + python install.py --continue_on_fail 2024-08-20T21:52:44.6561979Z checking packages torch, torchvision, torchaudio are installed...OK 2024-08-20T21:52:59.5822397Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch...OK 2024-08-20T21:53:01.2279980Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Background_Matting...OK 2024-08-20T21:53:03.1329593Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/LearningToPaint...OK 2024-08-20T21:53:04.8870477Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/Super_SloMo...OK 2024-08-20T21:53:04.9177249Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/alexnet...OK 2024-08-20T21:53:11.1849823Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_edgecnn...OK 2024-08-20T21:53:13.6868898Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gcn...OK 2024-08-20T21:53:16.1849089Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_gin...OK 2024-08-20T21:53:18.6833766Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/basic_gnn_sage...OK 2024-08-20T21:53:18.6835668Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/cm3leon_generate...SKIP - No install.py is found 2024-08-20T21:53:19.7766832Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dcgan...OK 2024-08-20T21:53:22.0439148Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/demucs...OK 2024-08-20T21:53:22.0742331Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/densenet121...OK 2024-08-20T21:54:05.6892774Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_c4...OK 2024-08-20T21:54:13.3719070Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_dc5...OK 2024-08-20T21:54:19.7284621Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_101_fpn...OK 2024-08-20T21:54:25.7929029Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_c4...OK 2024-08-20T21:54:33.3970823Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_dc5...OK 2024-08-20T21:54:39.7826264Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fasterrcnn_r_50_fpn...OK 2024-08-20T21:54:46.4391089Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_fcos_r_50_fpn...OK 2024-08-20T21:54:52.6027466Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn...OK 2024-08-20T21:54:58.9507537Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_c4...OK 2024-08-20T21:55:06.0561330Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_101_fpn...OK 2024-08-20T21:55:12.1725705Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_c4...OK 2024-08-20T21:55:18.3127113Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/detectron2_maskrcnn_r_50_fpn...OK 2024-08-20T21:55:19.5183295Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/dlrm...OK 2024-08-20T21:55:52.2338536Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_det_predictor...OK 2024-08-20T21:55:56.7694249Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/doctr_reco_predictor...OK 2024-08-20T21:56:01.8843981Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/drq...OK 2024-08-20T22:01:47.2179802Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/fastNLP_Bert...OK 2024-08-20T22:01:48.4221409Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_dp_cifar10...OK 2024-08-20T22:01:49.8911089Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/functorch_maml_omniglot...OK 2024-08-20T22:01:56.1147103Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Albert...OK 2024-08-20T22:02:02.9245049Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bart...OK 2024-08-20T22:02:08.1321129Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert...OK 2024-08-20T22:02:15.6135631Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Bert_large...OK 2024-08-20T22:02:21.2574375Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_BigBird...OK 2024-08-20T22:02:26.1762535Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_DistilBert...OK 2024-08-20T22:02:32.1181242Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2...OK 2024-08-20T22:02:48.0747792Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_GPT2_large...OK 2024-08-20T22:02:54.0120737Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Longformer...OK 2024-08-20T22:02:57.6913851Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Reformer...OK 2024-08-20T22:03:02.1903118Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5...OK 2024-08-20T22:03:08.7191318Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_base...OK 2024-08-20T22:03:08.7194633Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_generate...SKIP - No install.py is found 2024-08-20T22:03:21.4851924Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_T5_large...OK 2024-08-20T22:03:25.1621307Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_Whisper...OK 2024-08-20T22:03:25.1623691Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_clip...SKIP - No install.py is found 2024-08-20T22:03:32.5244718Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/hf_distil_whisper...OK 2024-08-20T22:03:33.7725940Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/lennard_jones...OK 2024-08-20T22:03:35.0431530Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama...OK 2024-08-20T22:04:19.3123369Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llama_v2_7b_16h...OK 2024-08-20T22:05:46.8576247Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/llava...OK 2024-08-20T22:05:46.8889547Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml...OK 2024-08-20T22:05:48.3742080Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/maml_omniglot...OK 2024-08-20T22:05:48.4053047Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mnasnet1_0...OK 2024-08-20T22:05:48.4341018Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2...OK 2024-08-20T22:05:48.4616031Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v2_quantized_qat...OK 2024-08-20T22:05:48.4885487Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/mobilenet_v3_large...OK 2024-08-20T22:05:48.5159943Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moco...OK 2024-08-20T22:06:09.3628110Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/moondream...OK 2024-08-20T22:06:09.3631438Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nanogpt...SKIP - No install.py is found 2024-08-20T22:06:10.6260828Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/nvidia_deeprecommender...OK 2024-08-20T22:06:13.0019684Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/opacus_cifar10...OK 2024-08-20T22:06:13.0329999Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_densenet...OK 2024-08-20T22:06:13.0618779Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/phlippe_resnet...OK 2024-08-20T22:06:13.0896957Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_equation_of_state...OK 2024-08-20T22:06:13.1168654Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_isoneutral_mixing...OK 2024-08-20T22:06:13.1440102Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pyhpc_turbulent_kinetic_energy...OK 2024-08-20T22:06:18.7741689Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_CycleGAN_and_pix2pix...OK 2024-08-20T22:06:20.2423158Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_stargan...OK 2024-08-20T22:06:23.5915121Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/pytorch_unet...OK 2024-08-20T22:06:23.6222080Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet152...OK 2024-08-20T22:06:23.6507343Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet18...OK 2024-08-20T22:06:23.6781932Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50...OK 2024-08-20T22:06:23.7055240Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnet50_quantized_qat...OK 2024-08-20T22:06:23.7324196Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/resnext50_32x4d...OK 2024-08-20T22:06:34.2649303Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam...OK 2024-08-20T22:06:46.2917399Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/sam_fast...OK 2024-08-20T22:06:46.3225685Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/shufflenet_v2_x1_0...OK 2024-08-20T22:06:46.3228706Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt...SKIP - No install.py is found 2024-08-20T22:06:46.3230534Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/simple_gpt_tp_manual...SKIP - No install.py is found 2024-08-20T22:06:48.5813055Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/soft_actor_critic...OK 2024-08-20T22:06:50.1627259Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/speech_transformer...OK 2024-08-20T22:06:50.1937494Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/squeezenet1_1...OK 2024-08-20T22:07:14.9340952Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_text_encoder...OK 2024-08-20T22:07:29.0901634Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/stable_diffusion_unet...OK 2024-08-20T22:07:33.7335912Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tacotron2...OK 2024-08-20T22:07:40.1695882Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientdet...OK 2024-08-20T22:07:40.2007168Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_efficientnet...OK 2024-08-20T22:07:40.2295748Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_nfnet...OK 2024-08-20T22:07:40.2569166Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_regnet...OK 2024-08-20T22:07:40.2840186Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_resnest...OK 2024-08-20T22:07:40.3108900Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer...OK 2024-08-20T22:07:40.3379951Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vision_transformer_large...OK 2024-08-20T22:07:40.3646683Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/timm_vovnet...OK 2024-08-20T22:07:49.7119070Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/torch_multimodal_clip...OK 2024-08-20T22:08:06.9908873Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/tts_angular...OK 2024-08-20T22:08:07.0222292Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vgg16...OK 2024-08-20T22:08:09.1415120Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/vision_maskrcnn...OK 2024-08-20T22:08:11.0932813Z running setup for /var/lib/jenkins/workspace/torchbench/torchbenchmark/models/yolov3...OK 2024-08-20T22:08:11.4863218Z + echo 'Print all dependencies after TorchBench is installed' 2024-08-20T22:08:11.4863907Z Print all dependencies after TorchBench is installed 2024-08-20T22:08:11.4864443Z + python -mpip freeze 2024-08-20T22:08:11.9529620Z absl-py==2.1.0 2024-08-20T22:08:11.9530209Z accelerate==0.33.0 2024-08-20T22:08:11.9530546Z aiohappyeyeballs==2.4.0 2024-08-20T22:08:11.9530923Z aiohttp==3.10.5 2024-08-20T22:08:11.9531213Z aiosignal==1.3.1 2024-08-20T22:08:11.9531507Z alabaster==0.7.13 2024-08-20T22:08:11.9531931Z annotated-types==0.7.0 2024-08-20T22:08:11.9532391Z antlr4-python3-runtime==4.9.3 2024-08-20T22:08:11.9532753Z anyascii==0.3.2 2024-08-20T22:08:11.9533029Z astroid==3.2.4 2024-08-20T22:08:11.9534721Z asttokens==2.4.1 2024-08-20T22:08:11.9535082Z astunparse==1.6.3 2024-08-20T22:08:11.9535534Z async-timeout==4.0.3 2024-08-20T22:08:11.9535831Z attrs==23.1.0 2024-08-20T22:08:11.9536110Z audioread==3.0.1 2024-08-20T22:08:11.9536651Z babel==2.16.0 2024-08-20T22:08:11.9536907Z backcall==0.2.0 2024-08-20T22:08:11.9537206Z beautifulsoup4==4.12.3 2024-08-20T22:08:11.9538375Z -e git+https://github.com/pytorch/benchmark@23512dbebd44a11eb84afbf53c3c071dd105297e#egg=bert_pytorch&subdirectory=torchbenchmark/models/BERT_pytorch 2024-08-20T22:08:11.9539548Z bibtexparser==2.0.0b7 2024-08-20T22:08:11.9539846Z black==24.8.0 2024-08-20T22:08:11.9540118Z blinker==1.8.2 2024-08-20T22:08:11.9540395Z blis==0.7.11 2024-08-20T22:08:11.9540664Z blobfile==2.1.1 2024-08-20T22:08:11.9540929Z bokeh==3.1.1 2024-08-20T22:08:11.9541189Z boto3==1.19.12 2024-08-20T22:08:11.9541476Z botocore==1.22.12 2024-08-20T22:08:11.9541758Z breathe==4.34.0 2024-08-20T22:08:11.9542029Z bs4==0.0.1 2024-08-20T22:08:11.9542271Z cachetools==5.5.0 2024-08-20T22:08:11.9542564Z cardboardlint==1.3.1 2024-08-20T22:08:11.9542869Z catalogue==2.0.10 2024-08-20T22:08:11.9543139Z certifi==2024.7.4 2024-08-20T22:08:11.9543612Z cffi==1.17.0 2024-08-20T22:08:11.9543960Z charset-normalizer==3.3.2 2024-08-20T22:08:11.9544290Z click==8.1.7 2024-08-20T22:08:11.9544560Z clldutils==3.22.2 2024-08-20T22:08:11.9544839Z cloudpathlib==0.18.1 2024-08-20T22:08:11.9545152Z cloudpickle==3.0.0 2024-08-20T22:08:11.9545443Z colorama==0.4.6 2024-08-20T22:08:11.9545708Z colorlog==6.8.2 2024-08-20T22:08:11.9545992Z comm==0.2.2 2024-08-20T22:08:11.9546250Z confection==0.1.5 2024-08-20T22:08:11.9546539Z contourpy==1.1.1 2024-08-20T22:08:11.9546817Z coremltools==5.0b5 2024-08-20T22:08:11.9547114Z cryptography==43.0.0 2024-08-20T22:08:11.9547419Z csvw==3.3.0 2024-08-20T22:08:11.9547673Z cycler==0.12.1 2024-08-20T22:08:11.9547930Z cymem==2.0.8 2024-08-20T22:08:11.9548219Z DALL-E==0.1 2024-08-20T22:08:11.9548475Z datasets==2.21.0 2024-08-20T22:08:11.9548740Z debugpy==1.8.5 2024-08-20T22:08:11.9549012Z decorator==5.1.1 2024-08-20T22:08:11.9549295Z defusedxml==0.7.1 2024-08-20T22:08:11.9549581Z Deprecated==1.2.14 2024-08-20T22:08:11.9550305Z detectron2 @ git+https://github.com/facebookresearch/detectron2.git@1a4df4d954089534daebbd7a0a45f08ece87056e 2024-08-20T22:08:11.9551133Z diffusers==0.20.2 2024-08-20T22:08:11.9551416Z dill==0.3.8 2024-08-20T22:08:11.9551664Z diskcache==5.6.3 2024-08-20T22:08:11.9551949Z distro==1.9.0 2024-08-20T22:08:11.9552221Z dlinfo==1.2.1 2024-08-20T22:08:11.9552493Z dnspython==2.6.1 2024-08-20T22:08:11.9552818Z docker-pycreds==0.4.0 2024-08-20T22:08:11.9553130Z docutils==0.16 2024-08-20T22:08:11.9553414Z dominate==2.9.1 2024-08-20T22:08:11.9554246Z effdet @ git+https://github.com/rwightman/efficientdet-pytorch.git@d43c9e34cd62d22b4205831bb735f6dd83b8e881 2024-08-20T22:08:11.9555050Z einops==0.8.0 2024-08-20T22:08:11.9555336Z exceptiongroup==1.2.2 2024-08-20T22:08:11.9555651Z execnet==2.1.1 2024-08-20T22:08:11.9555916Z executing==2.0.1 2024-08-20T22:08:11.9556202Z exhale==0.2.3 2024-08-20T22:08:11.9556476Z expecttest==0.1.6 2024-08-20T22:08:11.9556777Z fastjsonschema==2.20.0 2024-08-20T22:08:11.9557080Z FastNLP==0.6.0 2024-08-20T22:08:11.9557398Z ffmpeg-python==0.2.0 2024-08-20T22:08:11.9557713Z filelock==3.13.1 2024-08-20T22:08:11.9557977Z Flask==3.0.3 2024-08-20T22:08:11.9558242Z flatbuffers==2.0 2024-08-20T22:08:11.9558523Z fonttools==4.53.1 2024-08-20T22:08:11.9558806Z frozenlist==1.4.1 2024-08-20T22:08:11.9559078Z fsspec==2024.6.1 2024-08-20T22:08:11.9559357Z ftfy==6.2.3 2024-08-20T22:08:11.9560129Z functorch @ git+https://github.com/pytorch/functorch.git@b71aa0b4387b86c278132209b99538be48ef4c74 2024-08-20T22:08:11.9560868Z future==1.0.0 2024-08-20T22:08:11.9561137Z fvcore==0.1.5.post20221221 2024-08-20T22:08:11.9561470Z gdown==5.2.0 2024-08-20T22:08:11.9561736Z geojson==2.5.0 2024-08-20T22:08:11.9561994Z ghstack==0.8.0 2024-08-20T22:08:11.9562266Z gitdb==4.0.11 2024-08-20T22:08:11.9562536Z GitPython==3.1.43 2024-08-20T22:08:11.9562866Z google-auth==2.34.0 2024-08-20T22:08:11.9563206Z google-auth-oauthlib==1.0.0 2024-08-20T22:08:11.9563550Z greenlet==3.0.3 2024-08-20T22:08:11.9563827Z grpcio==1.65.5 2024-08-20T22:08:11.9564080Z gym==0.26.2 2024-08-20T22:08:11.9564467Z gym-notices==0.0.8 2024-08-20T22:08:11.9564752Z h5py==3.11.0 2024-08-20T22:08:11.9565020Z higher==0.2.1 2024-08-20T22:08:11.9565323Z huggingface-hub==0.24.6 2024-08-20T22:08:11.9565677Z hydra-core==1.3.2 2024-08-20T22:08:11.9565965Z hypothesis==5.35.1 2024-08-20T22:08:11.9566236Z idna==3.7 2024-08-20T22:08:11.9566490Z imageio==2.35.1 2024-08-20T22:08:11.9566772Z imagesize==1.4.1 2024-08-20T22:08:11.9567070Z importlib_metadata==8.3.0 2024-08-20T22:08:11.9567401Z importlib_resources==6.4.3 2024-08-20T22:08:11.9567731Z inflect==7.3.1 2024-08-20T22:08:11.9568012Z iniconfig==2.0.0 2024-08-20T22:08:11.9568283Z iopath==0.1.9 2024-08-20T22:08:11.9568554Z ipykernel==6.29.5 2024-08-20T22:08:11.9568838Z ipython==8.12.0 2024-08-20T22:08:11.9569118Z isodate==0.6.1 2024-08-20T22:08:11.9569373Z isort==5.13.2 2024-08-20T22:08:11.9569650Z itsdangerous==2.2.0 2024-08-20T22:08:11.9569938Z jedi==0.19.1 2024-08-20T22:08:11.9570184Z Jinja2==3.1.4 2024-08-20T22:08:11.9570454Z jmespath==0.10.0 2024-08-20T22:08:11.9570734Z joblib==1.4.2 2024-08-20T22:08:11.9571007Z jsonpatch==1.33 2024-08-20T22:08:11.9571278Z jsonpointer==3.0.0 2024-08-20T22:08:11.9571573Z jsonschema==4.23.0 2024-08-20T22:08:11.9571966Z jsonschema-specifications==2023.12.1 2024-08-20T22:08:11.9572358Z junitparser==2.1.1 2024-08-20T22:08:11.9572694Z jupyter-cache==0.6.1 2024-08-20T22:08:11.9573012Z jupyter_client==8.6.2 2024-08-20T22:08:11.9573323Z jupyter_core==5.7.2 2024-08-20T22:08:11.9573633Z kaldi-io==0.9.8 2024-08-20T22:08:11.9573912Z kiwisolver==1.4.5 2024-08-20T22:08:11.9574192Z kornia==0.7.3 2024-08-20T22:08:11.9574466Z kornia_rs==0.1.5 2024-08-20T22:08:11.9574732Z lameenc==1.7.0 2024-08-20T22:08:11.9575004Z langcodes==3.4.0 2024-08-20T22:08:11.9575288Z langdetect==1.0.9 2024-08-20T22:08:11.9575607Z language-tags==1.2.0 2024-08-20T22:08:11.9575914Z language_data==1.2.0 2024-08-20T22:08:11.9576208Z lark==0.12.0 2024-08-20T22:08:11.9576469Z libcst==1.1.0 2024-08-20T22:08:11.9576725Z librosa==0.9.2 2024-08-20T22:08:11.9577007Z lintrunner==0.12.5 2024-08-20T22:08:11.9577303Z llvmlite==0.32.1 2024-08-20T22:08:11.9577570Z lxml==4.9.4 2024-08-20T22:08:11.9577863Z marisa-trie==1.2.0 2024-08-20T22:08:11.9578153Z Markdown==3.7 2024-08-20T22:08:11.9578464Z markdown-it-py==2.2.0 2024-08-20T22:08:11.9578765Z MarkupSafe==2.1.5 2024-08-20T22:08:11.9579184Z matplotlib==3.5.3 2024-08-20T22:08:11.9579546Z matplotlib-inline==0.1.7 2024-08-20T22:08:11.9579864Z mccabe==0.7.0 2024-08-20T22:08:11.9580256Z mdit-py-plugins==0.3.5 2024-08-20T22:08:11.9580557Z mdurl==0.1.2 2024-08-20T22:08:11.9580861Z ml-dtypes==0.2.0 2024-08-20T22:08:11.9581154Z MonkeyType==23.3.0 2024-08-20T22:08:11.9581490Z more-itertools==10.4.0 2024-08-20T22:08:11.9581802Z mpmath==1.3.0 2024-08-20T22:08:11.9582079Z multidict==6.0.5 2024-08-20T22:08:11.9582375Z multiprocess==0.70.16 2024-08-20T22:08:11.9582678Z murmurhash==1.0.10 2024-08-20T22:08:11.9582967Z musdb==0.4.2 2024-08-20T22:08:11.9583241Z museval==0.4.1 2024-08-20T22:08:11.9583514Z mypy==1.10.0 2024-08-20T22:08:11.9583823Z mypy-extensions==1.0.0 2024-08-20T22:08:11.9584180Z myst-nb==0.17.2 2024-08-20T22:08:11.9584498Z myst-parser==0.18.1 2024-08-20T22:08:11.9584782Z nbclient==0.7.4 2024-08-20T22:08:11.9585062Z nbformat==5.10.4 2024-08-20T22:08:11.9585388Z nest-asyncio==1.6.0 2024-08-20T22:08:11.9585692Z networkx==2.8.8 2024-08-20T22:08:11.9585958Z ninja==1.10.2 2024-08-20T22:08:11.9586333Z nose==1.3.7 2024-08-20T22:08:11.9586594Z numba==0.49.0 2024-08-20T22:08:11.9586848Z numpy==1.21.2 2024-08-20T22:08:11.9587182Z nvidia-cublas-cu12==12.1.3.1 2024-08-20T22:08:11.9587600Z nvidia-cuda-cupti-cu12==12.1.105 2024-08-20T22:08:11.9588041Z nvidia-cuda-nvrtc-cu12==12.1.105 2024-08-20T22:08:11.9588472Z nvidia-cuda-runtime-cu12==12.1.105 2024-08-20T22:08:11.9588910Z nvidia-cudnn-cu12==9.1.0.70 2024-08-20T22:08:11.9589305Z nvidia-cufft-cu12==11.0.2.54 2024-08-20T22:08:11.9589709Z nvidia-curand-cu12==10.3.2.106 2024-08-20T22:08:11.9590118Z nvidia-cusolver-cu12==11.4.5.107 2024-08-20T22:08:11.9590552Z nvidia-cusparse-cu12==12.1.0.106 2024-08-20T22:08:11.9590975Z nvidia-nccl-cu12==2.20.5 2024-08-20T22:08:11.9591534Z nvidia-nvjitlink-cu12==12.6.20 2024-08-20T22:08:11.9591932Z nvidia-nvtx-cu12==12.1.105 2024-08-20T22:08:11.9592260Z oauthlib==3.2.2 2024-08-20T22:08:11.9592538Z omegaconf==2.3.0 2024-08-20T22:08:11.9592802Z onnx==1.16.1 2024-08-20T22:08:11.9593078Z onnxscript==0.1.0.dev20240817 2024-08-20T22:08:11.9593416Z opacus==1.5.2 2024-08-20T22:08:11.9593714Z opencv-python==4.8.0.74 2024-08-20T22:08:11.9594223Z opt-einsum==3.3.0 2024-08-20T22:08:11.9610187Z optree==0.12.1 2024-08-20T22:08:11.9610511Z packaging==24.1 2024-08-20T22:08:11.9610799Z pandas==2.0.3 2024-08-20T22:08:11.9611059Z parso==0.8.4 2024-08-20T22:08:11.9611329Z patch==1.16 2024-08-20T22:08:11.9611593Z pathspec==0.12.1 2024-08-20T22:08:11.9611867Z pexpect==4.9.0 2024-08-20T22:08:11.9612147Z phonemizer==3.3.0 2024-08-20T22:08:11.9612443Z pickleshare==0.7.5 2024-08-20T22:08:11.9612732Z pillow==10.3.0 2024-08-20T22:08:11.9613012Z pkgutil_resolve_name==1.3.10 2024-08-20T22:08:11.9613367Z platformdirs==4.2.2 2024-08-20T22:08:11.9613685Z pluggy==1.5.0 2024-08-20T22:08:11.9613938Z pooch==1.8.2 2024-08-20T22:08:11.9614212Z portalocker==2.10.1 2024-08-20T22:08:11.9614507Z preshed==3.0.9 2024-08-20T22:08:11.9614788Z prettytable==3.11.0 2024-08-20T22:08:11.9615077Z prompt_toolkit==3.0.47 2024-08-20T22:08:11.9615388Z protobuf==3.20.2 2024-08-20T22:08:11.9615680Z psutil==6.0.0 2024-08-20T22:08:11.9615941Z ptyprocess==0.7.0 2024-08-20T22:08:11.9616231Z pure_eval==0.2.3 2024-08-20T22:08:11.9616644Z py-cpuinfo==9.0.0 2024-08-20T22:08:11.9616929Z pyaml==24.7.0 2024-08-20T22:08:11.9617186Z pyarrow==17.0.0 2024-08-20T22:08:11.9617466Z pyasn1==0.6.0 2024-08-20T22:08:11.9617741Z pyasn1_modules==0.4.0 2024-08-20T22:08:11.9618046Z pyclipper==1.3.0.post5 2024-08-20T22:08:11.9618364Z pycocotools==2.0.7 2024-08-20T22:08:11.9618656Z pycparser==2.22 2024-08-20T22:08:11.9619070Z pycryptodomex==3.20.0 2024-08-20T22:08:11.9619371Z pydantic==2.8.2 2024-08-20T22:08:11.9619659Z pydantic_core==2.20.1 2024-08-20T22:08:11.9619983Z pydot==3.0.1 2024-08-20T22:08:11.9620243Z pygame==2.6.0 2024-08-20T22:08:11.9620516Z PyGithub==2.3.0 2024-08-20T22:08:11.9620796Z Pygments==2.15.0 2024-08-20T22:08:11.9621076Z PyJWT==2.9.0 2024-08-20T22:08:11.9621332Z pylatexenc==2.10 2024-08-20T22:08:11.9621614Z pylint==3.2.6 2024-08-20T22:08:11.9621880Z PyNaCl==1.5.0 2024-08-20T22:08:11.9622140Z pynvml==11.5.3 2024-08-20T22:08:11.9622421Z pyparsing==3.1.2 2024-08-20T22:08:11.9622709Z pypdfium2==4.30.0 2024-08-20T22:08:11.9622994Z pysbd==0.3.4 2024-08-20T22:08:11.9623245Z PySocks==1.7.1 2024-08-20T22:08:11.9623517Z pytest==7.3.2 2024-08-20T22:08:11.9623849Z pytest-benchmark==4.0.0 2024-08-20T22:08:11.9624200Z pytest-cpp==2.3.0 2024-08-20T22:08:11.9624543Z pytest-flakefinder==1.1.0 2024-08-20T22:08:11.9624936Z pytest-rerunfailures==14.0 2024-08-20T22:08:11.9625313Z pytest-xdist==3.3.1 2024-08-20T22:08:11.9625653Z python-dateutil==2.9.0.post0 2024-08-20T22:08:11.9626485Z python-doctr @ git+https://github.com/xuzhao9/doctr.git@7f8a3e61bd6480c4e7b7cde47af0aa2153309c56 2024-08-20T22:08:11.9627270Z python-etcd==0.4.5 2024-08-20T22:08:11.9628354Z pytorch-labs-segment-anything-fast @ git+https://github.com/pytorch-labs/segment-anything-fast.git@de861af9badd03e8b40f5e063d70754a3dc6b4f4 2024-08-20T22:08:11.9630199Z -e git+https://github.com/pytorch/pytorch_sphinx_theme.git@05ee809ecf4c396cce8f44a172484f2946eb255a#egg=pytorch_sphinx_theme 2024-08-20T22:08:11.9631098Z pytz==2024.1 2024-08-20T22:08:11.9631378Z PyWavelets==1.4.1 2024-08-20T22:08:11.9631740Z PyYAML @ file:///croot/pyyaml_1698096049011/work 2024-08-20T22:08:11.9632168Z pyzmq==26.1.1 2024-08-20T22:08:11.9632445Z rapidfuzz==3.9.6 2024-08-20T22:08:11.9632725Z rdflib==7.0.0 2024-08-20T22:08:11.9632990Z redis==5.0.8 2024-08-20T22:08:11.9633247Z referencing==0.35.1 2024-08-20T22:08:11.9633542Z regex==2024.7.24 2024-08-20T22:08:11.9633819Z requests==2.32.3 2024-08-20T22:08:11.9634142Z requests-oauthlib==2.0.0 2024-08-20T22:08:11.9634472Z resampy==0.3.1 2024-08-20T22:08:11.9634741Z rfc3986==1.5.0 2024-08-20T22:08:11.9635005Z rich==13.7.1 2024-08-20T22:08:11.9635337Z rockset==1.0.3 2024-08-20T22:08:11.9635643Z rpds-py==0.20.0 2024-08-20T22:08:11.9635917Z rsa==4.9 2024-08-20T22:08:11.9636158Z s3transfer==0.5.2 2024-08-20T22:08:11.9636450Z safetensors==0.4.4 2024-08-20T22:08:11.9636780Z scikit-image==0.19.3 2024-08-20T22:08:11.9637118Z scikit-learn==1.3.2 2024-08-20T22:08:11.9637402Z scipy==1.10.1 2024-08-20T22:08:11.9637669Z segments==2.2.1 2024-08-20T22:08:11.9637953Z sentencepiece==0.2.0 2024-08-20T22:08:11.9638278Z sentry-sdk==2.13.0 2024-08-20T22:08:11.9638571Z setproctitle==1.3.3 2024-08-20T22:08:11.9638867Z shapely==2.0.6 2024-08-20T22:08:11.9639142Z shellingham==1.5.4 2024-08-20T22:08:11.9639424Z simplejson==3.19.3 2024-08-20T22:08:11.9639803Z six @ file:///tmp/build/80754af9/six_1644875935023/work 2024-08-20T22:08:11.9640301Z smart-open==7.0.4 2024-08-20T22:08:11.9640571Z smmap==5.0.1 2024-08-20T22:08:11.9640849Z snowballstemmer==2.2.0 2024-08-20T22:08:11.9641174Z sortedcontainers==2.4.0 2024-08-20T22:08:11.9641601Z soundfile==0.12.1 2024-08-20T22:08:11.9641972Z soupsieve==2.6 2024-08-20T22:08:11.9642226Z spacy==3.7.6 2024-08-20T22:08:11.9642514Z spacy-legacy==3.0.12 2024-08-20T22:08:11.9642819Z spacy-loggers==1.0.5 2024-08-20T22:08:11.9643098Z Sphinx==5.3.0 2024-08-20T22:08:11.9643537Z sphinx-copybutton==0.5.0 2024-08-20T22:08:11.9644070Z sphinx-panels==0.4.1 2024-08-20T22:08:11.9644516Z sphinxcontrib-applehelp==1.0.4 2024-08-20T22:08:11.9644999Z sphinxcontrib-devhelp==1.0.2 2024-08-20T22:08:11.9645423Z sphinxcontrib-htmlhelp==2.0.1 2024-08-20T22:08:11.9645841Z sphinxcontrib-jsmath==1.0.1 2024-08-20T22:08:11.9646232Z sphinxcontrib-katex==0.8.6 2024-08-20T22:08:11.9646627Z sphinxcontrib-qthelp==1.0.3 2024-08-20T22:08:11.9647068Z sphinxcontrib-serializinghtml==1.1.5 2024-08-20T22:08:11.9647467Z SQLAlchemy==2.0.32 2024-08-20T22:08:11.9647756Z srsly==2.4.8 2024-08-20T22:08:11.9648054Z stack-data==0.6.3 2024-08-20T22:08:11.9648339Z stempeg==0.2.3 2024-08-20T22:08:11.9648600Z submitit==1.5.1 2024-08-20T22:08:11.9648875Z sympy==1.12.1 2024-08-20T22:08:11.9649147Z tabulate==0.9.0 2024-08-20T22:08:11.9649461Z tb-nightly==2.13.0a20230426 2024-08-20T22:08:11.9649796Z tensorboard==2.13.0 2024-08-20T22:08:11.9650159Z tensorboard-data-server==0.7.2 2024-08-20T22:08:11.9650527Z tensorboardX==2.6.2.2 2024-08-20T22:08:11.9650823Z termcolor==2.4.0 2024-08-20T22:08:11.9651101Z thinc==8.2.5 2024-08-20T22:08:11.9651379Z threadpoolctl==3.5.0 2024-08-20T22:08:11.9651674Z tifffile==2023.7.10 2024-08-20T22:08:11.9652512Z timm @ git+https://github.com/huggingface/pytorch-image-models.git@730b907b4d45a4713cbc425cbf224c46089fd514 2024-08-20T22:08:11.9653327Z tlparse==0.3.25 2024-08-20T22:08:11.9653608Z tokenizers==0.15.2 2024-08-20T22:08:11.9653881Z tomli==2.0.1 2024-08-20T22:08:11.9654145Z tomlkit==0.13.2 2024-08-20T22:08:11.9655422Z torch @ file:///var/lib/jenkins/workspace/dist/torch-2.5.0a0%2Bgit40ec5f6-cp38-cp38-linux_x86_64.whl#sha256=af3cdc81a563c847a86d810c93c14c6dabe8cf56a0cb55c23bba116e1c3deca6 2024-08-20T22:08:11.9657220Z torch_geometric @ git+https://github.com/pyg-team/pytorch_geometric.git@cabcd4097442ba60aa1efa11e1619dd9bb8fb527 2024-08-20T22:08:11.9658251Z torchao @ git+https://github.com/pytorch/ao.git@2c8e3f368613f896fde14cc24b9268c9e125575e 2024-08-20T22:08:11.9659436Z torchaudio @ git+https://github.com/pytorch/audio.git@b3f6f511f2a1082bd56b13a3f6794e7fc3ba4862 2024-08-20T22:08:11.9660631Z torchdata @ git+https://github.com/pytorch/data.git@11bb5b847ea8b9e0d9bb82db3304daf383008d3f 2024-08-20T22:08:11.9661820Z torchmultimodal @ git+https://github.com/facebookresearch/multimodal.git@e4d288b45b89cee462a21ab264405f3f368adc21 2024-08-20T22:08:11.9662980Z torchtext @ git+https://github.com/pytorch/text.git@b0ebddc648d279826089db91775375221777a2db 2024-08-20T22:08:11.9664011Z torchvision @ git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:08:11.9664730Z tornado==6.4.1 2024-08-20T22:08:11.9665008Z tqdm==4.66.5 2024-08-20T22:08:11.9665277Z traitlets==5.14.3 2024-08-20T22:08:11.9665558Z transformers==4.38.1 2024-08-20T22:08:11.9665945Z treetable==0.2.5 2024-08-20T22:08:11.9666799Z -e git+https://github.com/openai/triton@dedb7bdf339a3546896d4820366ca562c586bfa0#egg=triton&subdirectory=python 2024-08-20T22:08:11.9667609Z typeguard==4.3.0 2024-08-20T22:08:11.9667893Z typer==0.12.4 2024-08-20T22:08:11.9668212Z typing-inspect==0.9.0 2024-08-20T22:08:11.9668543Z typing_extensions==4.12.2 2024-08-20T22:08:11.9668857Z tzdata==2024.1 2024-08-20T22:08:11.9669249Z Unidecode==1.3.8 2024-08-20T22:08:11.9669607Z unittest-xml-reporting==3.2.0 2024-08-20T22:08:11.9669970Z uritemplate==4.1.1 2024-08-20T22:08:11.9670249Z urllib3==1.26.19 2024-08-20T22:08:11.9670527Z visdom==0.2.4 2024-08-20T22:08:11.9670794Z wandb==0.17.7 2024-08-20T22:08:11.9671044Z wasabi==1.1.3 2024-08-20T22:08:11.9671307Z wcwidth==0.2.13 2024-08-20T22:08:11.9671583Z weasel==0.4.1 2024-08-20T22:08:11.9671899Z websocket-client==1.8.0 2024-08-20T22:08:11.9672206Z Werkzeug==3.0.3 2024-08-20T22:08:11.9672481Z wrapt==1.16.0 2024-08-20T22:08:11.9672746Z xdoctest==1.1.0 2024-08-20T22:08:11.9673011Z xxhash==3.5.0 2024-08-20T22:08:11.9673284Z xyzservices==2024.6.0 2024-08-20T22:08:11.9673587Z yacs==0.1.8 2024-08-20T22:08:11.9673841Z yarl==1.9.4 2024-08-20T22:08:11.9674119Z z3-solver==4.12.2.0 2024-08-20T22:08:11.9674403Z zipp==3.20.0 2024-08-20T22:08:11.9860524Z + popd 2024-08-20T22:08:11.9860810Z ~/workspace 2024-08-20T22:08:11.9861257Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing != *cpu* ]] 2024-08-20T22:08:11.9861782Z ++ pwd 2024-08-20T22:08:11.9862120Z + PYTHONPATH=/var/lib/jenkins/workspace/torchbench 2024-08-20T22:08:11.9862607Z + test_dynamo_benchmark torchbench 0 2024-08-20T22:08:11.9865695Z ++ pwd 2024-08-20T22:08:11.9867598Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:08:11.9868137Z + local suite=torchbench 2024-08-20T22:08:11.9868449Z + shift 2024-08-20T22:08:11.9868697Z + local shard_id=0 2024-08-20T22:08:11.9868980Z + shift 2024-08-20T22:08:11.9869418Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *perf_compare* ]] 2024-08-20T22:08:11.9870141Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *perf* ]] 2024-08-20T22:08:11.9870812Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *cpu* ]] 2024-08-20T22:08:11.9871330Z + local dt=float32 2024-08-20T22:08:11.9871761Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *amp* ]] 2024-08-20T22:08:11.9872265Z + dt=amp 2024-08-20T22:08:11.9872691Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *freezing* ]] 2024-08-20T22:08:11.9873575Z + test_single_dynamo_benchmark inference torchbench 0 --inference --amp --freezing 2024-08-20T22:08:11.9874193Z ++ pwd 2024-08-20T22:08:11.9874637Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:08:11.9875320Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:08:11.9889063Z + local name=inference 2024-08-20T22:08:11.9889428Z + shift 2024-08-20T22:08:11.9889680Z + local suite=torchbench 2024-08-20T22:08:11.9890003Z + shift 2024-08-20T22:08:11.9890255Z + local shard_id=0 2024-08-20T22:08:11.9890538Z + shift 2024-08-20T22:08:11.9890798Z + partition_flags=() 2024-08-20T22:08:11.9891115Z + local partition_flags 2024-08-20T22:08:11.9891521Z + [[ -n 2 ]] 2024-08-20T22:08:11.9891806Z + [[ -n 0 ]] 2024-08-20T22:08:11.9892417Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T22:08:11.9893430Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *perf_compare* ]] 2024-08-20T22:08:11.9894163Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *perf* ]] 2024-08-20T22:08:11.9894865Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *aot_inductor* ]] 2024-08-20T22:08:11.9895638Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing != *cpu_aot_inductor* ]] 2024-08-20T22:08:11.9896374Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *_avx2* ]] 2024-08-20T22:08:11.9897069Z + [[ dynamic_cpu_aot_inductor_torchbench_amp_freezing == *_avx512* ]] 2024-08-20T22:08:11.9899423Z + python benchmarks/dynamo/torchbench.py --ci --accuracy --timing --explain --export-aot-inductor --dynamic-shapes --dynamic-batch-only --device cpu --inference --amp --freezing --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv 2024-08-20T22:08:17.0941101Z 2024-08-20T22:08:19.0530185Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:08:19.0533135Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:08:19.0534206Z cpu eval BERT_pytorch 2024-08-20T22:08:19.4248401Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:19.8411167Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:20.3320543Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:20.9698408Z ERROR:common: 2024-08-20T22:08:20.9699170Z Traceback (most recent call last): 2024-08-20T22:08:20.9699979Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:08:20.9700964Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:08:20.9701782Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:08:20.9702821Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:08:20.9703694Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:08:20.9704418Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:08:20.9705617Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:08:20.9706518Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:08:20.9707466Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:08:20.9708282Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:08:20.9709317Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:08:20.9710233Z return self._call_impl(*args, **kwargs) 2024-08-20T22:08:20.9711204Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:08:20.9712057Z return forward_call(*args, **kwargs) 2024-08-20T22:08:20.9712948Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:08:20.9713726Z return fn(*args, **kwargs) 2024-08-20T22:08:20.9714691Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:08:20.9715593Z return self._call_impl(*args, **kwargs) 2024-08-20T22:08:20.9716559Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:08:20.9717415Z return forward_call(*args, **kwargs) 2024-08-20T22:08:20.9718375Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:08:20.9719238Z return self._torchdynamo_orig_callable( 2024-08-20T22:08:20.9720393Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:08:20.9721222Z return _compile( 2024-08-20T22:08:20.9722070Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:08:20.9723040Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:08:20.9724142Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:08:20.9725101Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:08:20.9726143Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:08:20.9727109Z return function(*args, **kwargs) 2024-08-20T22:08:20.9728066Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:08:20.9729026Z out_code = transform_code_object(code, transform) 2024-08-20T22:08:20.9730206Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:08:20.9731244Z transformations(instructions, code_options) 2024-08-20T22:08:20.9732186Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:08:20.9732988Z return fn(*args, **kwargs) 2024-08-20T22:08:20.9733884Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:08:20.9734702Z tracer.run() 2024-08-20T22:08:20.9735510Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:08:20.9736316Z super().run() 2024-08-20T22:08:20.9737132Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:08:20.9737936Z while self.step(): 2024-08-20T22:08:20.9738773Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:08:20.9739744Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:08:20.9740754Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:08:20.9741599Z return inner_fn(self, inst) 2024-08-20T22:08:20.9742566Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:08:20.9743649Z self.call_function(fn, args, {}) 2024-08-20T22:08:20.9744643Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:08:20.9745806Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:08:20.9747026Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:08:20.9747961Z return super().call_function(tx, args, kwargs) 2024-08-20T22:08:20.9749030Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:08:20.9749988Z return super().call_function(tx, args, kwargs) 2024-08-20T22:08:20.9751047Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:08:20.9752145Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:08:20.9753436Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:08:20.9754611Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:08:20.9755909Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:08:20.9756971Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:08:20.9758486Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:08:20.9759345Z tracer.run() 2024-08-20T22:08:20.9760162Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:08:20.9760975Z while self.step(): 2024-08-20T22:08:20.9761811Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:08:20.9762688Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:08:20.9763771Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:08:20.9764617Z return inner_fn(self, inst) 2024-08-20T22:08:20.9765577Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:08:20.9766472Z self.call_function(fn, args, {}) 2024-08-20T22:08:20.9767456Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:08:20.9768599Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:08:20.9769794Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:08:20.9770826Z return super().call_function(tx, args, kwargs) 2024-08-20T22:08:20.9771863Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:08:20.9772792Z return super().call_function(tx, args, kwargs) 2024-08-20T22:08:20.9773822Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:08:20.9774899Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:08:20.9776158Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:08:20.9777472Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:08:20.9778674Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:08:20.9779693Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:08:20.9780764Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:08:20.9781633Z tracer.run() 2024-08-20T22:08:20.9782447Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:08:20.9783256Z while self.step(): 2024-08-20T22:08:20.9784097Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:08:20.9784976Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:08:20.9786000Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1739, in STORE_ATTR 2024-08-20T22:08:20.9786840Z assert ( 2024-08-20T22:08:20.9787245Z AssertionError: Mutating module attribute mask during export. 2024-08-20T22:08:20.9787672Z 2024-08-20T22:08:20.9787775Z from user code: 2024-08-20T22:08:20.9788669Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch/bert_pytorch/model/bert.py", line 50, in forward 2024-08-20T22:08:20.9789672Z x = transformer.forward(x, mask) 2024-08-20T22:08:20.9790708Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch/bert_pytorch/model/transformer.py", line 47, in forward 2024-08-20T22:08:20.9791856Z self.lambda_module.set_mask(mask) 2024-08-20T22:08:20.9792955Z File "/var/lib/jenkins/workspace/torchbench/torchbenchmark/models/BERT_pytorch/bert_pytorch/model/transformer.py", line 16, in set_mask 2024-08-20T22:08:20.9793961Z self.mask = mask 2024-08-20T22:08:20.9794149Z 2024-08-20T22:08:20.9794433Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:08:20.9794891Z 2024-08-20T22:08:20.9795175Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:08:20.9795725Z fail_to_run 2024-08-20T22:08:20.9796321Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:20.9797002Z TIMING: 2024-08-20T22:08:20.9797750Z STATS: call_* op count: 0 | FakeTensor.__torch_dispatch__:12 | FakeTensorMode.__torch_dispatch__:53 | attempt fast:2 | fast is_contiguous:2 2024-08-20T22:08:20.9799850Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:08:24.0343920Z 2024-08-20T22:08:27.7621138Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:08:27.7621643Z loading model: 0it [00:03, ?it/s] 2024-08-20T22:08:27.7622097Z cpu eval Background_Matting 2024-08-20T22:08:27.7629020Z pass_due_to_skip 2024-08-20T22:08:27.7630659Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:27.7631374Z TIMING: 2024-08-20T22:08:27.7631643Z STATS: call_* op count: 0 2024-08-20T22:08:27.7632173Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:08:30.8627411Z 2024-08-20T22:08:33.0106742Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:08:33.0107254Z loading model: 0it [00:02, ?it/s] 2024-08-20T22:08:33.0107718Z cpu eval LearningToPaint 2024-08-20T22:08:33.1042978Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:33.1422346Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:08:33.1790954Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:09:05.6827352Z pass 2024-08-20T22:09:05.6830285Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:09:05.6832387Z TIMING: backend_compile:9e-05 entire_frame_compile:1.34795 total_wall_time:1.34795 2024-08-20T22:09:05.6835154Z STATS: call_* op count: 72 | FakeTensor.__torch_dispatch__:389 | FakeTensorMode.__torch_dispatch__:9321 | attempt fast:240 | fast is_contiguous:208 | ProxyTorchDispatchMode.__torch_dispatch__:1234 | slow no contiguity match:32 2024-08-20T22:09:05.6837944Z Dynamo produced 1 graphs covering 72 ops with 0 graph breaks (0 unique) 2024-08-20T22:09:09.2540334Z 2024-08-20T22:09:10.5331830Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/vgg16-397923af.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/vgg16-397923af.pth 2024-08-20T22:09:10.5517623Z 2024-08-20T22:09:10.5517630Z 2024-08-20T22:09:10.6523009Z 0% 0.00/528M [00:00::value && (((M < 32) && (M != 16)) || ((N < 32) && (N != 16)))), int>::type = 0; int64_t = long int]’: 2024-08-20T22:15:13.9874218Z /tmp/tmp3ydr6m14/clhhunrq4n7kt4gvmpqqtyy734a4chbakclns45tfeolz7v2qyps/cejkiksc2hbcuk35qb7k7sf5ck3m5qytzpoadiep4mgbovjm2njt.cpp:789:59: required from here 2024-08-20T22:15:13.9876508Z /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/include/ATen/cpu/vec/vec512/vec512_bfloat16.h:1395:27: warning: left shift count >= width of type [-Wshift-count-overflow] 2024-08-20T22:15:13.9877889Z 1395 | __mmask32 dst_mask = (1 << M) - 1; 2024-08-20T22:15:13.9878461Z | ~~~^~~~~ 2024-08-20T22:15:34.9089481Z pass 2024-08-20T22:15:34.9091777Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:15:34.9093423Z TIMING: backend_compile:9e-05 entire_frame_compile:3.65647 total_wall_time:3.65647 2024-08-20T22:15:34.9095047Z STATS: call_* op count: 67 | FakeTensorMode.__torch_dispatch__:33088 | FakeTensor.__torch_dispatch__:536 | ProxyTorchDispatchMode.__torch_dispatch__:1341 2024-08-20T22:15:34.9096582Z Dynamo produced 1 graphs covering 67 ops with 0 graph breaks (0 unique) 2024-08-20T22:15:38.4260119Z 2024-08-20T22:15:38.6822373Z loading model: 0it [00:00, ?it/s]Downloading: "https://download.pytorch.org/models/densenet121-a639ec97.pth" to /var/lib/jenkins/.cache/torch/hub/checkpoints/densenet121-a639ec97.pth 2024-08-20T22:15:38.7007826Z 2024-08-20T22:15:38.7007835Z 2024-08-20T22:15:38.8012992Z 0% 0.00/30.8M [00:00 2024-08-20T22:19:38.9872558Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:19:38.9873643Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:19:38.9874545Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:19:38.9875554Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:19:38.9876360Z return node.target(*args, **kwargs) 2024-08-20T22:19:38.9877420Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:19:38.9878386Z r = self.shape_env.evaluate_expr( 2024-08-20T22:19:38.9879375Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:19:38.9880256Z return retlog(event.run(self)) 2024-08-20T22:19:38.9881184Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:19:38.9882031Z return self.f(*args, **kwargs) 2024-08-20T22:19:38.9883090Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:19:38.9884434Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:19:38.9885907Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:19:38.9886878Z raise self._make_data_dependent_error( 2024-08-20T22:19:38.9888218Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:19:38.9889786Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:19:38.9890432Z 2024-08-20T22:19:38.9890721Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:19:38.9891780Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:19:38.9892695Z return node.target(*args, **kwargs) 2024-08-20T22:19:38.9892994Z 2024-08-20T22:19:38.9893182Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:19:38.9893966Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:19:38.9911287Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:19:38.9912640Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:19:38.9913483Z 2024-08-20T22:19:38.9913617Z User Stack (most recent call last): 2024-08-20T22:19:38.9914041Z (snipped, see stack below for prefix) 2024-08-20T22:19:38.9915094Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:19:38.9916005Z return self.inference(batched_inputs) 2024-08-20T22:19:38.9917014Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:19:38.9917949Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:19:38.9919167Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:19:38.9920082Z images = ImageList.from_tensors( 2024-08-20T22:19:38.9921079Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:19:38.9922363Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:19:38.9922781Z 2024-08-20T22:19:38.9923046Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:19:38.9923457Z 2024-08-20T22:19:38.9923747Z During handling of the above exception, another exception occurred: 2024-08-20T22:19:38.9924290Z 2024-08-20T22:19:38.9924424Z Traceback (most recent call last): 2024-08-20T22:19:38.9925159Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:19:38.9925934Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:19:38.9926735Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:19:38.9927704Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:19:38.9928746Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:19:38.9929453Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:19:38.9930462Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:19:38.9931348Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:19:38.9932274Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:19:38.9933082Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:19:38.9934093Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:19:38.9934995Z return self._call_impl(*args, **kwargs) 2024-08-20T22:19:38.9935939Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:19:38.9936765Z return forward_call(*args, **kwargs) 2024-08-20T22:19:38.9937640Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:19:38.9938700Z return fn(*args, **kwargs) 2024-08-20T22:19:38.9939689Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:19:38.9940605Z return self._call_impl(*args, **kwargs) 2024-08-20T22:19:38.9941578Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:19:38.9942447Z return forward_call(*args, **kwargs) 2024-08-20T22:19:38.9943587Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:19:38.9944451Z return self._torchdynamo_orig_callable( 2024-08-20T22:19:38.9945431Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:19:38.9946256Z return _compile( 2024-08-20T22:19:38.9947116Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:19:38.9948094Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:19:38.9949202Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:19:38.9950172Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:19:38.9951238Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:19:38.9952098Z return function(*args, **kwargs) 2024-08-20T22:19:38.9953198Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:19:38.9954148Z out_code = transform_code_object(code, transform) 2024-08-20T22:19:38.9955342Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:19:38.9956426Z transformations(instructions, code_options) 2024-08-20T22:19:38.9957310Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:19:38.9958059Z return fn(*args, **kwargs) 2024-08-20T22:19:38.9958901Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:19:38.9959758Z tracer.run() 2024-08-20T22:19:38.9960519Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:19:38.9961274Z super().run() 2024-08-20T22:19:38.9962049Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:19:38.9962811Z while self.step(): 2024-08-20T22:19:38.9963592Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:19:38.9964414Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:19:38.9965346Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:19:38.9966131Z return inner_fn(self, inst) 2024-08-20T22:19:38.9967014Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:19:38.9967849Z self.call_function(fn, args, {}) 2024-08-20T22:19:38.9968769Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:19:38.9969838Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:19:38.9970958Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:19:38.9971834Z return super().call_function(tx, args, kwargs) 2024-08-20T22:19:38.9972833Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:19:38.9973884Z return super().call_function(tx, args, kwargs) 2024-08-20T22:19:38.9974941Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:19:38.9976013Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:19:38.9977290Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:19:38.9978712Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:19:38.9979944Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:19:38.9980901Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:19:38.9981965Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:19:38.9982834Z tracer.run() 2024-08-20T22:19:38.9983657Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:19:38.9984474Z while self.step(): 2024-08-20T22:19:38.9985313Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:19:38.9986202Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:19:38.9987207Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:19:38.9988146Z return inner_fn(self, inst) 2024-08-20T22:19:38.9989108Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:19:38.9989996Z self.call_function(fn, args, {}) 2024-08-20T22:19:38.9991082Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:19:38.9992151Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:19:38.9993260Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:19:38.9994153Z return super().call_function(tx, args, kwargs) 2024-08-20T22:19:38.9995214Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:19:38.9996098Z return super().call_function(tx, args, kwargs) 2024-08-20T22:19:38.9997097Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:19:38.9998117Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:19:38.9999331Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:19:39.0000432Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:19:39.0001556Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:19:39.0002432Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:19:39.0003433Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:19:39.0004238Z tracer.run() 2024-08-20T22:19:39.0005007Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:19:39.0005751Z while self.step(): 2024-08-20T22:19:39.0006545Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:19:39.0007363Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:19:39.0008295Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:19:39.0009080Z return inner_fn(self, inst) 2024-08-20T22:19:39.0009982Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:19:39.0010846Z self.call_function(fn, args, kwargs) 2024-08-20T22:19:39.0011780Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:19:39.0012852Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:19:39.0013975Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:19:39.0014862Z return super().call_function(tx, args, kwargs) 2024-08-20T22:19:39.0016017Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:19:39.0017116Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:19:39.0018448Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:19:39.0019802Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:19:39.0021021Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:19:39.0021976Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:19:39.0023128Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:19:39.0023984Z tracer.run() 2024-08-20T22:19:39.0024816Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:19:39.0025639Z while self.step(): 2024-08-20T22:19:39.0026495Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:19:39.0027365Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:19:39.0028378Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:19:39.0029292Z return inner_fn(self, inst) 2024-08-20T22:19:39.0030206Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:19:39.0031230Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:19:39.0032284Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:19:39.0033117Z return handler(tx, args, kwargs) 2024-08-20T22:19:39.0034129Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:19:39.0035026Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:19:39.0035952Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:19:39.0036903Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:19:39.0037991Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:19:39.0039017Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:19:39.0040045Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:19:39.0040821Z raise UserError( # noqa: B904 2024-08-20T22:19:39.0043529Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:19:39.0046101Z 2024-08-20T22:19:39.0046378Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:19:39.0047457Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:19:39.0048274Z return node.target(*args, **kwargs) 2024-08-20T22:19:39.0048567Z 2024-08-20T22:19:39.0048759Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:19:39.0049573Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:19:39.0050570Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:19:39.0051814Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:19:39.0052651Z 2024-08-20T22:19:39.0052789Z User Stack (most recent call last): 2024-08-20T22:19:39.0053217Z (snipped, see stack below for prefix) 2024-08-20T22:19:39.0054239Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:19:39.0055153Z return self.inference(batched_inputs) 2024-08-20T22:19:39.0056399Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:19:39.0057277Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:19:39.0058666Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:19:39.0059638Z images = ImageList.from_tensors( 2024-08-20T22:19:39.0060672Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:19:39.0061775Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:19:39.0062210Z 2024-08-20T22:19:39.0062462Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:19:39.0063673Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:19:39.0064585Z 2024-08-20T22:19:39.0064705Z from user code: 2024-08-20T22:19:39.0065592Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:19:39.0066504Z return self.inference(batched_inputs) 2024-08-20T22:19:39.0067543Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:19:39.0068502Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:19:39.0069622Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:19:39.0070563Z images = ImageList.from_tensors( 2024-08-20T22:19:39.0071639Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:19:39.0072671Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:19:39.0073061Z 2024-08-20T22:19:39.0073349Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:19:39.0073770Z 2024-08-20T22:19:39.0074044Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:19:39.0074569Z fail_to_run 2024-08-20T22:19:39.0075145Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:19:39.0075812Z TIMING: 2024-08-20T22:19:39.0076332Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:48 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:19:39.0077167Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:19:42.1194528Z 2024-08-20T22:19:57.7383902Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:19:57.7384422Z loading model: 0it [00:15, ?it/s] 2024-08-20T22:19:57.7384883Z cpu eval detectron2_fasterrcnn_r_101_dc5 2024-08-20T22:20:16.9802422Z WARNING:common:fp64 golden ref were not generated for detectron2_fasterrcnn_r_101_dc5. Setting accuracy check to cosine 2024-08-20T22:20:17.0203588Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:20:38.5252350Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:21:00.0601745Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:21:11.3809758Z W0820 22:21:11.380077 15843 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Eq(u0, 800), hint=None, expect_rational=True, size_oblivious=True, forcing_spec=False 2024-08-20T22:21:11.3812140Z E0820 22:21:11.380232 15843 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Eq(u0, 800), None), **{'fx_node': eq_3, 'size_oblivious': True}) 2024-08-20T22:21:11.4162464Z ERROR:common: 2024-08-20T22:21:11.4162947Z Traceback (most recent call last): 2024-08-20T22:21:11.4163970Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:21:11.4164830Z return node.target(*args, **kwargs) 2024-08-20T22:21:11.4165918Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:21:11.4167128Z r = self.shape_env.evaluate_expr( 2024-08-20T22:21:11.4168145Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:21:11.4169037Z return retlog(event.run(self)) 2024-08-20T22:21:11.4169982Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:21:11.4170830Z return self.f(*args, **kwargs) 2024-08-20T22:21:11.4171889Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:21:11.4173234Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:21:11.4174846Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:21:11.4175842Z raise self._make_data_dependent_error( 2024-08-20T22:21:11.4177209Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:21:11.4178418Z 2024-08-20T22:21:11.4178695Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:21:11.4179755Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:21:11.4180566Z return node.target(*args, **kwargs) 2024-08-20T22:21:11.4180859Z 2024-08-20T22:21:11.4181144Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:21:11.4181950Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:21:11.4182947Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:21:11.4184175Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:21:11.4185028Z 2024-08-20T22:21:11.4185163Z User Stack (most recent call last): 2024-08-20T22:21:11.4185592Z (snipped, see stack below for prefix) 2024-08-20T22:21:11.4186599Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:21:11.4187510Z return self.inference(batched_inputs) 2024-08-20T22:21:11.4188517Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:21:11.4189469Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:21:11.4190583Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:21:11.4191535Z images = ImageList.from_tensors( 2024-08-20T22:21:11.4192533Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:21:11.4193649Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:21:11.4194080Z 2024-08-20T22:21:11.4194331Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:21:11.4194741Z 2024-08-20T22:21:11.4195044Z The above exception was the direct cause of the following exception: 2024-08-20T22:21:11.4195499Z 2024-08-20T22:21:11.4195631Z Traceback (most recent call last): 2024-08-20T22:21:11.4196551Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1999, in get_fake_value 2024-08-20T22:21:11.4197387Z ret_val = wrap_fake_exception( 2024-08-20T22:21:11.4198335Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1484, in wrap_fake_exception 2024-08-20T22:21:11.4199172Z return fn() 2024-08-20T22:21:11.4199935Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2000, in 2024-08-20T22:21:11.4200912Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:21:11.4201921Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:21:11.4202823Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:21:11.4203807Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:21:11.4204619Z return node.target(*args, **kwargs) 2024-08-20T22:21:11.4205694Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:21:11.4206656Z r = self.shape_env.evaluate_expr( 2024-08-20T22:21:11.4207684Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:21:11.4208670Z return retlog(event.run(self)) 2024-08-20T22:21:11.4209608Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:21:11.4210467Z return self.f(*args, **kwargs) 2024-08-20T22:21:11.4211501Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:21:11.4212842Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:21:11.4214302Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:21:11.4215290Z raise self._make_data_dependent_error( 2024-08-20T22:21:11.4216575Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:21:11.4218126Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:21:11.4218848Z 2024-08-20T22:21:11.4219126Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:21:11.4220177Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:21:11.4220990Z return node.target(*args, **kwargs) 2024-08-20T22:21:11.4221283Z 2024-08-20T22:21:11.4221489Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:21:11.4222282Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:21:11.4223268Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:21:11.4224500Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:21:11.4225343Z 2024-08-20T22:21:11.4225491Z User Stack (most recent call last): 2024-08-20T22:21:11.4225909Z (snipped, see stack below for prefix) 2024-08-20T22:21:11.4226923Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:21:11.4227832Z return self.inference(batched_inputs) 2024-08-20T22:21:11.4228852Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:21:11.4229787Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:21:11.4230894Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:21:11.4231840Z images = ImageList.from_tensors( 2024-08-20T22:21:11.4232854Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:21:11.4233950Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:21:11.4234382Z 2024-08-20T22:21:11.4234629Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:21:11.4235049Z 2024-08-20T22:21:11.4235416Z During handling of the above exception, another exception occurred: 2024-08-20T22:21:11.4235890Z 2024-08-20T22:21:11.4236025Z Traceback (most recent call last): 2024-08-20T22:21:11.4236762Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:21:11.4237523Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:21:11.4238333Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:21:11.4239310Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:21:11.4240186Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:21:11.4240972Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:21:11.4241995Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:21:11.4242895Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:21:11.4243983Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:21:11.4244810Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:21:11.4245847Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:21:11.4246773Z return self._call_impl(*args, **kwargs) 2024-08-20T22:21:11.4247747Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:21:11.4248601Z return forward_call(*args, **kwargs) 2024-08-20T22:21:11.4249481Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:21:11.4250271Z return fn(*args, **kwargs) 2024-08-20T22:21:11.4251233Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:21:11.4252157Z return self._call_impl(*args, **kwargs) 2024-08-20T22:21:11.4253115Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:21:11.4253975Z return forward_call(*args, **kwargs) 2024-08-20T22:21:11.4254926Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:21:11.4255791Z return self._torchdynamo_orig_callable( 2024-08-20T22:21:11.4256826Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:21:11.4257625Z return _compile( 2024-08-20T22:21:11.4258506Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:21:11.4259647Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:21:11.4260759Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:21:11.4261733Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:21:11.4262790Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:21:11.4263647Z return function(*args, **kwargs) 2024-08-20T22:21:11.4264602Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:21:11.4265539Z out_code = transform_code_object(code, transform) 2024-08-20T22:21:11.4266715Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:21:11.4267751Z transformations(instructions, code_options) 2024-08-20T22:21:11.4268704Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:21:11.4269499Z return fn(*args, **kwargs) 2024-08-20T22:21:11.4270536Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:21:11.4271369Z tracer.run() 2024-08-20T22:21:11.4272293Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:21:11.4273148Z super().run() 2024-08-20T22:21:11.4273910Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:21:11.4274668Z while self.step(): 2024-08-20T22:21:11.4275452Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:21:11.4276250Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:21:11.4277272Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:21:11.4278061Z return inner_fn(self, inst) 2024-08-20T22:21:11.4278961Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:21:11.4279783Z self.call_function(fn, args, {}) 2024-08-20T22:21:11.4280699Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:21:11.4281769Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:21:11.4282892Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:21:11.4283763Z return super().call_function(tx, args, kwargs) 2024-08-20T22:21:11.4284750Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:21:11.4285635Z return super().call_function(tx, args, kwargs) 2024-08-20T22:21:11.4286626Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:21:11.4287660Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:21:11.4288849Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:21:11.4289942Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:21:11.4291060Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:21:11.4291948Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:21:11.4292928Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:21:11.4293737Z tracer.run() 2024-08-20T22:21:11.4294496Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:21:11.4295251Z while self.step(): 2024-08-20T22:21:11.4296027Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:21:11.4296839Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:21:11.4297760Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:21:11.4298795Z return inner_fn(self, inst) 2024-08-20T22:21:11.4299765Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:21:11.4300646Z self.call_function(fn, args, {}) 2024-08-20T22:21:11.4301631Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:21:11.4302787Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:21:11.4303993Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:21:11.4304998Z return super().call_function(tx, args, kwargs) 2024-08-20T22:21:11.4306073Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:21:11.4307027Z return super().call_function(tx, args, kwargs) 2024-08-20T22:21:11.4308089Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:21:11.4309190Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:21:11.4310491Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:21:11.4311805Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:21:11.4312982Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:21:11.4313922Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:21:11.4315053Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:21:11.4315851Z tracer.run() 2024-08-20T22:21:11.4316809Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:21:11.4317594Z while self.step(): 2024-08-20T22:21:11.4318405Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:21:11.4319266Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:21:11.4320240Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:21:11.4321066Z return inner_fn(self, inst) 2024-08-20T22:21:11.4322016Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:21:11.4322915Z self.call_function(fn, args, kwargs) 2024-08-20T22:21:11.4323889Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:21:11.4325006Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:21:11.4326167Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:21:11.4327101Z return super().call_function(tx, args, kwargs) 2024-08-20T22:21:11.4328244Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:21:11.4329276Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:21:11.4330683Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:21:11.4331815Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:21:11.4332988Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:21:11.4333918Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:21:11.4334956Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:21:11.4335787Z tracer.run() 2024-08-20T22:21:11.4336587Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:21:11.4337383Z while self.step(): 2024-08-20T22:21:11.4338280Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:21:11.4339308Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:21:11.4340386Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:21:11.4341238Z return inner_fn(self, inst) 2024-08-20T22:21:11.4342139Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:21:11.4343087Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:21:11.4344343Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:21:11.4345242Z return handler(tx, args, kwargs) 2024-08-20T22:21:11.4346326Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:21:11.4347401Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:21:11.4348391Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:21:11.4349420Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:21:11.4350591Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:21:11.4351698Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:21:11.4352803Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:21:11.4353635Z raise UserError( # noqa: B904 2024-08-20T22:21:11.4356483Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:21:11.4358657Z 2024-08-20T22:21:11.4358913Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:21:11.4359890Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:21:11.4360643Z return node.target(*args, **kwargs) 2024-08-20T22:21:11.4360914Z 2024-08-20T22:21:11.4361091Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:21:11.4361837Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:21:11.4362751Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:21:11.4363896Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:21:11.4364672Z 2024-08-20T22:21:11.4364808Z User Stack (most recent call last): 2024-08-20T22:21:11.4365194Z (snipped, see stack below for prefix) 2024-08-20T22:21:11.4366125Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:21:11.4366968Z return self.inference(batched_inputs) 2024-08-20T22:21:11.4367910Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:21:11.4368778Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:21:11.4369805Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:21:11.4370681Z images = ImageList.from_tensors( 2024-08-20T22:21:11.4371611Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:21:11.4372623Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:21:11.4373020Z 2024-08-20T22:21:11.4373251Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:21:11.4374435Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:21:11.4375206Z 2024-08-20T22:21:11.4375317Z from user code: 2024-08-20T22:21:11.4376141Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:21:11.4376975Z return self.inference(batched_inputs) 2024-08-20T22:21:11.4377917Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:21:11.4379071Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:21:11.4380192Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:21:11.4382501Z images = ImageList.from_tensors( 2024-08-20T22:21:11.4383527Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:21:11.4384647Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:21:11.4385064Z 2024-08-20T22:21:11.4385375Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:21:11.4385832Z 2024-08-20T22:21:11.4386139Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:21:11.4386696Z fail_to_run 2024-08-20T22:21:11.4387306Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:21:11.4388018Z TIMING: 2024-08-20T22:21:11.4388579Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:48 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:21:11.4389476Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:21:14.5816660Z 2024-08-20T22:21:24.3213943Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:21:24.3214483Z loading model: 0it [00:09, ?it/s] 2024-08-20T22:21:24.3215045Z cpu eval detectron2_fasterrcnn_r_101_fpn 2024-08-20T22:21:42.8951476Z WARNING:common:fp64 golden ref were not generated for detectron2_fasterrcnn_r_101_fpn. Setting accuracy check to cosine 2024-08-20T22:21:42.9267515Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:21:55.3251762Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:22:07.7372947Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:22:14.3456255Z W0820 22:22:14.344660 16170 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Eq(u0, 800), hint=None, expect_rational=True, size_oblivious=True, forcing_spec=False 2024-08-20T22:22:14.3458783Z E0820 22:22:14.344841 16170 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Eq(u0, 800), None), **{'fx_node': eq_3, 'size_oblivious': True}) 2024-08-20T22:22:14.3724941Z ERROR:common: 2024-08-20T22:22:14.3725297Z Traceback (most recent call last): 2024-08-20T22:22:14.3726320Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:22:14.3727142Z return node.target(*args, **kwargs) 2024-08-20T22:22:14.3728222Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:22:14.3729197Z r = self.shape_env.evaluate_expr( 2024-08-20T22:22:14.3730191Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:22:14.3731084Z return retlog(event.run(self)) 2024-08-20T22:22:14.3732008Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:22:14.3732871Z return self.f(*args, **kwargs) 2024-08-20T22:22:14.3733923Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:22:14.3735492Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:22:14.3736967Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:22:14.3737965Z raise self._make_data_dependent_error( 2024-08-20T22:22:14.3739461Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:22:14.3740536Z 2024-08-20T22:22:14.3740809Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:22:14.3741992Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:22:14.3742796Z return node.target(*args, **kwargs) 2024-08-20T22:22:14.3743100Z 2024-08-20T22:22:14.3743541Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:22:14.3744364Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:22:14.3745345Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:14.3746586Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:22:14.3747441Z 2024-08-20T22:22:14.3747575Z User Stack (most recent call last): 2024-08-20T22:22:14.3747999Z (snipped, see stack below for prefix) 2024-08-20T22:22:14.3749003Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:22:14.3749921Z return self.inference(batched_inputs) 2024-08-20T22:22:14.3750937Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:22:14.3751883Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:22:14.3752988Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:22:14.3753936Z images = ImageList.from_tensors( 2024-08-20T22:22:14.3754949Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:22:14.3756059Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:22:14.3756477Z 2024-08-20T22:22:14.3756741Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:14.3757151Z 2024-08-20T22:22:14.3757440Z The above exception was the direct cause of the following exception: 2024-08-20T22:22:14.3757914Z 2024-08-20T22:22:14.3758045Z Traceback (most recent call last): 2024-08-20T22:22:14.3758965Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1999, in get_fake_value 2024-08-20T22:22:14.3759799Z ret_val = wrap_fake_exception( 2024-08-20T22:22:14.3760734Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1484, in wrap_fake_exception 2024-08-20T22:22:14.3761571Z return fn() 2024-08-20T22:22:14.3762351Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2000, in 2024-08-20T22:22:14.3763258Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:22:14.3764246Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:22:14.3765150Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:22:14.3766151Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:22:14.3766962Z return node.target(*args, **kwargs) 2024-08-20T22:22:14.3768033Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:22:14.3769104Z r = self.shape_env.evaluate_expr( 2024-08-20T22:22:14.3770098Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:22:14.3770978Z return retlog(event.run(self)) 2024-08-20T22:22:14.3771919Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:22:14.3772763Z return self.f(*args, **kwargs) 2024-08-20T22:22:14.3773811Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:22:14.3775152Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:22:14.3776708Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:22:14.3777682Z raise self._make_data_dependent_error( 2024-08-20T22:22:14.3779076Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:22:14.3780637Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:22:14.3781288Z 2024-08-20T22:22:14.3781573Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:22:14.3782613Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:22:14.3783409Z return node.target(*args, **kwargs) 2024-08-20T22:22:14.3783712Z 2024-08-20T22:22:14.3783907Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:22:14.3784718Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:22:14.3785708Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:14.3786936Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:22:14.3787783Z 2024-08-20T22:22:14.3787916Z User Stack (most recent call last): 2024-08-20T22:22:14.3788341Z (snipped, see stack below for prefix) 2024-08-20T22:22:14.3789356Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:22:14.3790272Z return self.inference(batched_inputs) 2024-08-20T22:22:14.3791278Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:22:14.3792233Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:22:14.3793344Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:22:14.3794294Z images = ImageList.from_tensors( 2024-08-20T22:22:14.3795301Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:22:14.3796412Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:22:14.3796845Z 2024-08-20T22:22:14.3797097Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:14.3797511Z 2024-08-20T22:22:14.3797811Z During handling of the above exception, another exception occurred: 2024-08-20T22:22:14.3798362Z 2024-08-20T22:22:14.3798505Z Traceback (most recent call last): 2024-08-20T22:22:14.3799200Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:22:14.3799953Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:22:14.3800747Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:22:14.3801689Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:22:14.3802603Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:22:14.3803324Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:22:14.3804323Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:22:14.3805204Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:22:14.3806106Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:22:14.3806912Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:22:14.3807911Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:22:14.3808869Z return self._call_impl(*args, **kwargs) 2024-08-20T22:22:14.3809801Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:22:14.3810639Z return forward_call(*args, **kwargs) 2024-08-20T22:22:14.3811697Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:22:14.3812481Z return fn(*args, **kwargs) 2024-08-20T22:22:14.3813431Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:22:14.3814354Z return self._call_impl(*args, **kwargs) 2024-08-20T22:22:14.3815321Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:22:14.3816179Z return forward_call(*args, **kwargs) 2024-08-20T22:22:14.3817113Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:22:14.3817981Z return self._torchdynamo_orig_callable( 2024-08-20T22:22:14.3819026Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:22:14.3819854Z return _compile( 2024-08-20T22:22:14.3820703Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:22:14.3821658Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:22:14.3822774Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:22:14.3823738Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:22:14.3824793Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:22:14.3825635Z return function(*args, **kwargs) 2024-08-20T22:22:14.3826616Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:22:14.3827553Z out_code = transform_code_object(code, transform) 2024-08-20T22:22:14.3828722Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:22:14.3829745Z transformations(instructions, code_options) 2024-08-20T22:22:14.3830807Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:22:14.3831548Z return fn(*args, **kwargs) 2024-08-20T22:22:14.3832384Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:22:14.3833135Z tracer.run() 2024-08-20T22:22:14.3833900Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:22:14.3834649Z super().run() 2024-08-20T22:22:14.3835413Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:22:14.3836153Z while self.step(): 2024-08-20T22:22:14.3836938Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:22:14.3837825Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:22:14.3838753Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:22:14.3839714Z return inner_fn(self, inst) 2024-08-20T22:22:14.3840643Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:22:14.3841515Z self.call_function(fn, args, {}) 2024-08-20T22:22:14.3842473Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:22:14.3843715Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:22:14.3845231Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:22:14.3846185Z return super().call_function(tx, args, kwargs) 2024-08-20T22:22:14.3847264Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:22:14.3848218Z return super().call_function(tx, args, kwargs) 2024-08-20T22:22:14.3849269Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:22:14.3850382Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:22:14.3851690Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:22:14.3852869Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:22:14.3854083Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:22:14.3855028Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:22:14.3856102Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:22:14.3857027Z tracer.run() 2024-08-20T22:22:14.3857782Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:22:14.3858748Z while self.step(): 2024-08-20T22:22:14.3859602Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:22:14.3860486Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:22:14.3861490Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:22:14.3862327Z return inner_fn(self, inst) 2024-08-20T22:22:14.3863289Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:22:14.3864181Z self.call_function(fn, args, {}) 2024-08-20T22:22:14.3865177Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:22:14.3866323Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:22:14.3867511Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:22:14.3868477Z return super().call_function(tx, args, kwargs) 2024-08-20T22:22:14.3869543Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:22:14.3870493Z return super().call_function(tx, args, kwargs) 2024-08-20T22:22:14.3871651Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:22:14.3872719Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:22:14.3874080Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:22:14.3875230Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:22:14.3876414Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:22:14.3877437Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:22:14.3878428Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:22:14.3879234Z tracer.run() 2024-08-20T22:22:14.3879990Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:22:14.3880787Z while self.step(): 2024-08-20T22:22:14.3881571Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:22:14.3882387Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:22:14.3883308Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:22:14.3884087Z return inner_fn(self, inst) 2024-08-20T22:22:14.3884984Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:22:14.3885832Z self.call_function(fn, args, kwargs) 2024-08-20T22:22:14.3886753Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:22:14.3887818Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:22:14.3888924Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:22:14.3889809Z return super().call_function(tx, args, kwargs) 2024-08-20T22:22:14.3890797Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:22:14.3891820Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:22:14.3893004Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:22:14.3894093Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:22:14.3895205Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:22:14.3896081Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:22:14.3897069Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:22:14.3897853Z tracer.run() 2024-08-20T22:22:14.3898856Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:22:14.3899674Z while self.step(): 2024-08-20T22:22:14.3900524Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:22:14.3901393Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:22:14.3902398Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:22:14.3903247Z return inner_fn(self, inst) 2024-08-20T22:22:14.3904149Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:22:14.3905084Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:22:14.3906213Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:22:14.3907111Z return handler(tx, args, kwargs) 2024-08-20T22:22:14.3908266Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:22:14.3909228Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:22:14.3910224Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:22:14.3911319Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:22:14.3912398Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:22:14.3913419Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:22:14.3914438Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:22:14.3915274Z raise UserError( # noqa: B904 2024-08-20T22:22:14.3917826Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:22:14.3919978Z 2024-08-20T22:22:14.3920234Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:22:14.3921204Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:22:14.3921944Z return node.target(*args, **kwargs) 2024-08-20T22:22:14.3922225Z 2024-08-20T22:22:14.3922402Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:22:14.3923151Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:22:14.3924062Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:14.3925200Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:22:14.3925987Z 2024-08-20T22:22:14.3926109Z User Stack (most recent call last): 2024-08-20T22:22:14.3926502Z (snipped, see stack below for prefix) 2024-08-20T22:22:14.3927433Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:22:14.3928277Z return self.inference(batched_inputs) 2024-08-20T22:22:14.3929206Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:22:14.3930086Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:22:14.3931122Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:22:14.3931998Z images = ImageList.from_tensors( 2024-08-20T22:22:14.3932921Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:22:14.3933944Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:22:14.3934340Z 2024-08-20T22:22:14.3934569Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:22:14.3935684Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:22:14.3936445Z 2024-08-20T22:22:14.3936553Z from user code: 2024-08-20T22:22:14.3937358Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:22:14.3938285Z return self.inference(batched_inputs) 2024-08-20T22:22:14.3939459Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:22:14.3940404Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:22:14.3941583Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:22:14.3942536Z images = ImageList.from_tensors( 2024-08-20T22:22:14.3943689Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:22:14.3944801Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:22:14.3945219Z 2024-08-20T22:22:14.3945512Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:22:14.3945980Z 2024-08-20T22:22:14.3946273Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:22:14.3946934Z fail_to_run 2024-08-20T22:22:14.3947548Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:22:14.3948254Z TIMING: 2024-08-20T22:22:14.3948827Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:51 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:22:14.3949729Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:22:17.5021078Z 2024-08-20T22:22:34.6541916Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:22:34.6542379Z loading model: 0it [00:17, ?it/s] 2024-08-20T22:22:34.6542821Z cpu eval detectron2_fasterrcnn_r_50_c4 2024-08-20T22:22:42.8291205Z WARNING:common:fp64 golden ref were not generated for detectron2_fasterrcnn_r_50_c4. Setting accuracy check to cosine 2024-08-20T22:22:42.8421508Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:12.0726043Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:41.4403347Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:56.4430239Z W0820 22:23:56.442495 16497 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Eq(u0, 800), hint=None, expect_rational=True, size_oblivious=True, forcing_spec=False 2024-08-20T22:23:56.4432649Z E0820 22:23:56.442659 16497 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Eq(u0, 800), None), **{'fx_node': eq_3, 'size_oblivious': True}) 2024-08-20T22:23:56.4690733Z ERROR:common: 2024-08-20T22:23:56.4691095Z Traceback (most recent call last): 2024-08-20T22:23:56.4692067Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:56.4692894Z return node.target(*args, **kwargs) 2024-08-20T22:23:56.4693986Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:23:56.4694972Z r = self.shape_env.evaluate_expr( 2024-08-20T22:23:56.4695966Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:23:56.4696840Z return retlog(event.run(self)) 2024-08-20T22:23:56.4697805Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:23:56.4698791Z return self.f(*args, **kwargs) 2024-08-20T22:23:56.4699858Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:23:56.4701205Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:23:56.4702665Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:23:56.4703668Z raise self._make_data_dependent_error( 2024-08-20T22:23:56.4705033Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:23:56.4706089Z 2024-08-20T22:23:56.4706613Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:23:56.4707671Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:56.4708474Z return node.target(*args, **kwargs) 2024-08-20T22:23:56.4708778Z 2024-08-20T22:23:56.4708968Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:23:56.4709772Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:23:56.4710757Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:56.4711984Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:23:56.4712960Z 2024-08-20T22:23:56.4727503Z User Stack (most recent call last): 2024-08-20T22:23:56.4728021Z (snipped, see stack below for prefix) 2024-08-20T22:23:56.4729122Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:56.4730049Z return self.inference(batched_inputs) 2024-08-20T22:23:56.4731125Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:56.4732072Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:56.4733173Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:56.4734123Z images = ImageList.from_tensors( 2024-08-20T22:23:56.4735134Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:56.4736252Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:56.4736670Z 2024-08-20T22:23:56.4736923Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:56.4737348Z 2024-08-20T22:23:56.4737644Z The above exception was the direct cause of the following exception: 2024-08-20T22:23:56.4738220Z 2024-08-20T22:23:56.4738357Z Traceback (most recent call last): 2024-08-20T22:23:56.4739288Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1999, in get_fake_value 2024-08-20T22:23:56.4740135Z ret_val = wrap_fake_exception( 2024-08-20T22:23:56.4741070Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1484, in wrap_fake_exception 2024-08-20T22:23:56.4741910Z return fn() 2024-08-20T22:23:56.4742690Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2000, in 2024-08-20T22:23:56.4743762Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:23:56.4744753Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:23:56.4745666Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:23:56.4746676Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:56.4747488Z return node.target(*args, **kwargs) 2024-08-20T22:23:56.4748546Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:23:56.4749503Z r = self.shape_env.evaluate_expr( 2024-08-20T22:23:56.4750486Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:23:56.4751361Z return retlog(event.run(self)) 2024-08-20T22:23:56.4752285Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:23:56.4753144Z return self.f(*args, **kwargs) 2024-08-20T22:23:56.4754184Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:23:56.4755691Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:23:56.4757162Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:23:56.4758133Z raise self._make_data_dependent_error( 2024-08-20T22:23:56.4759422Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:23:56.4760978Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:23:56.4761723Z 2024-08-20T22:23:56.4762007Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:23:56.4763055Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:56.4763860Z return node.target(*args, **kwargs) 2024-08-20T22:23:56.4764166Z 2024-08-20T22:23:56.4764360Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:23:56.4765174Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:23:56.4766167Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:56.4767390Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:23:56.4768238Z 2024-08-20T22:23:56.4768370Z User Stack (most recent call last): 2024-08-20T22:23:56.4768792Z (snipped, see stack below for prefix) 2024-08-20T22:23:56.4769802Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:56.4770702Z return self.inference(batched_inputs) 2024-08-20T22:23:56.4771724Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:56.4772680Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:56.4773797Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:56.4774734Z images = ImageList.from_tensors( 2024-08-20T22:23:56.4775741Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:56.4776851Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:56.4777267Z 2024-08-20T22:23:56.4777529Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:56.4777943Z 2024-08-20T22:23:56.4778313Z During handling of the above exception, another exception occurred: 2024-08-20T22:23:56.4778770Z 2024-08-20T22:23:56.4778904Z Traceback (most recent call last): 2024-08-20T22:23:56.4779632Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:23:56.4780493Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:23:56.4781312Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:23:56.4782270Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:23:56.4783137Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:23:56.4783877Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:23:56.4784904Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:23:56.4785793Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:23:56.4786740Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:23:56.4787570Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:23:56.4788685Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:23:56.4789607Z return self._call_impl(*args, **kwargs) 2024-08-20T22:23:56.4790569Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:23:56.4791435Z return forward_call(*args, **kwargs) 2024-08-20T22:23:56.4792429Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:23:56.4793194Z return fn(*args, **kwargs) 2024-08-20T22:23:56.4794122Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:23:56.4795086Z return self._call_impl(*args, **kwargs) 2024-08-20T22:23:56.4796024Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:23:56.4796862Z return forward_call(*args, **kwargs) 2024-08-20T22:23:56.4797780Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:23:56.4798626Z return self._torchdynamo_orig_callable( 2024-08-20T22:23:56.4799564Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:23:56.4800366Z return _compile( 2024-08-20T22:23:56.4801169Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:23:56.4802112Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:23:56.4803199Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:23:56.4804141Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:23:56.4805148Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:23:56.4805979Z return function(*args, **kwargs) 2024-08-20T22:23:56.4806923Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:23:56.4807834Z out_code = transform_code_object(code, transform) 2024-08-20T22:23:56.4808980Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:23:56.4809970Z transformations(instructions, code_options) 2024-08-20T22:23:56.4810902Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:23:56.4811684Z return fn(*args, **kwargs) 2024-08-20T22:23:56.4812568Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:23:56.4813354Z tracer.run() 2024-08-20T22:23:56.4814157Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:23:56.4814952Z super().run() 2024-08-20T22:23:56.4815745Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:56.4816523Z while self.step(): 2024-08-20T22:23:56.4817348Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:56.4818269Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:56.4819437Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:56.4820274Z return inner_fn(self, inst) 2024-08-20T22:23:56.4821236Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:23:56.4822136Z self.call_function(fn, args, {}) 2024-08-20T22:23:56.4823117Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:23:56.4824341Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:23:56.4825565Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:23:56.4826522Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:56.4827595Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:23:56.4828554Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:56.4829607Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:23:56.4832138Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:23:56.4833422Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:23:56.4834575Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:23:56.4835737Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:23:56.4836670Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:23:56.4837704Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:23:56.4838536Z tracer.run() 2024-08-20T22:23:56.4839315Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:56.4840110Z while self.step(): 2024-08-20T22:23:56.4840939Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:56.4841797Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:56.4842784Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:56.4843906Z return inner_fn(self, inst) 2024-08-20T22:23:56.4844880Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:23:56.4845774Z self.call_function(fn, args, {}) 2024-08-20T22:23:56.4846763Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:23:56.4847897Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:23:56.4849104Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:23:56.4850062Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:56.4851132Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:23:56.4852075Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:56.4853138Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:23:56.4854251Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:23:56.4855557Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:23:56.4856827Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:23:56.4857993Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:23:56.4859172Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:23:56.4860243Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:23:56.4861108Z tracer.run() 2024-08-20T22:23:56.4862027Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:56.4862851Z while self.step(): 2024-08-20T22:23:56.4863702Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:56.4864588Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:56.4865574Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:56.4866425Z return inner_fn(self, inst) 2024-08-20T22:23:56.4867417Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:23:56.4868431Z self.call_function(fn, args, kwargs) 2024-08-20T22:23:56.4869420Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:23:56.4870573Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:23:56.4871812Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:23:56.4872695Z return super().call_function(tx, args, kwargs) 2024-08-20T22:23:56.4873688Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:23:56.4874703Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:23:56.4875909Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:23:56.4877001Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:23:56.4878119Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:23:56.4878998Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:23:56.4879989Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:23:56.4881111Z tracer.run() 2024-08-20T22:23:56.4881954Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:23:56.4882759Z while self.step(): 2024-08-20T22:23:56.4883611Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:23:56.4884491Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:23:56.4885495Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:23:56.4886347Z return inner_fn(self, inst) 2024-08-20T22:23:56.4887229Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:23:56.4888178Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:23:56.4889302Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:23:56.4890195Z return handler(tx, args, kwargs) 2024-08-20T22:23:56.4891267Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:23:56.4892247Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:23:56.4893436Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:23:56.4894386Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:23:56.4895450Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:23:56.4896475Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:23:56.4897614Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:23:56.4898461Z raise UserError( # noqa: B904 2024-08-20T22:23:56.4901418Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:23:56.4903745Z 2024-08-20T22:23:56.4904098Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:23:56.4905136Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:23:56.4905952Z return node.target(*args, **kwargs) 2024-08-20T22:23:56.4906256Z 2024-08-20T22:23:56.4906452Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:23:56.4907256Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:23:56.4908243Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:56.4909465Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:23:56.4910312Z 2024-08-20T22:23:56.4910445Z User Stack (most recent call last): 2024-08-20T22:23:56.4910975Z (snipped, see stack below for prefix) 2024-08-20T22:23:56.4911913Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:56.4912745Z return self.inference(batched_inputs) 2024-08-20T22:23:56.4913688Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:56.4914570Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:56.4915596Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:56.4916457Z images = ImageList.from_tensors( 2024-08-20T22:23:56.4917388Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:56.4918412Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:56.4918796Z 2024-08-20T22:23:56.4919037Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:23:56.4920145Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:23:56.4920916Z 2024-08-20T22:23:56.4921012Z from user code: 2024-08-20T22:23:56.4921833Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:23:56.4922677Z return self.inference(batched_inputs) 2024-08-20T22:23:56.4923623Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:23:56.4924490Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:23:56.4925515Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:23:56.4926393Z images = ImageList.from_tensors( 2024-08-20T22:23:56.4927324Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:23:56.4928335Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:23:56.4928733Z 2024-08-20T22:23:56.4929002Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:23:56.4929437Z 2024-08-20T22:23:56.4929772Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:23:56.4930294Z fail_to_run 2024-08-20T22:23:56.4930861Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:23:56.4931515Z TIMING: 2024-08-20T22:23:56.4932047Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:48 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:23:56.4932874Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:23:59.5636343Z 2024-08-20T22:24:14.0062128Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:24:14.0062656Z loading model: 0it [00:14, ?it/s] 2024-08-20T22:24:14.0063110Z cpu eval detectron2_fasterrcnn_r_50_dc5 2024-08-20T22:24:28.0507612Z WARNING:common:fp64 golden ref were not generated for detectron2_fasterrcnn_r_50_dc5. Setting accuracy check to cosine 2024-08-20T22:24:28.1005541Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:24:48.6358055Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:25:09.1609449Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:25:20.1303572Z W0820 22:25:20.129786 16824 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Eq(u0, 800), hint=None, expect_rational=True, size_oblivious=True, forcing_spec=False 2024-08-20T22:25:20.1305921Z E0820 22:25:20.129983 16824 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Eq(u0, 800), None), **{'fx_node': eq_3, 'size_oblivious': True}) 2024-08-20T22:25:20.1658678Z ERROR:common: 2024-08-20T22:25:20.1659073Z Traceback (most recent call last): 2024-08-20T22:25:20.1661886Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:25:20.1662756Z return node.target(*args, **kwargs) 2024-08-20T22:25:20.1663884Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:25:20.1664847Z r = self.shape_env.evaluate_expr( 2024-08-20T22:25:20.1665848Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:25:20.1666752Z return retlog(event.run(self)) 2024-08-20T22:25:20.1667699Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:25:20.1668561Z return self.f(*args, **kwargs) 2024-08-20T22:25:20.1669608Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:25:20.1670968Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:25:20.1672456Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:25:20.1673444Z raise self._make_data_dependent_error( 2024-08-20T22:25:20.1674813Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:25:20.1675890Z 2024-08-20T22:25:20.1676166Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:25:20.1677220Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:25:20.1678033Z return node.target(*args, **kwargs) 2024-08-20T22:25:20.1678326Z 2024-08-20T22:25:20.1678530Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:25:20.1679326Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:25:20.1680313Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:25:20.1681781Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:25:20.1682637Z 2024-08-20T22:25:20.1682876Z User Stack (most recent call last): 2024-08-20T22:25:20.1683310Z (snipped, see stack below for prefix) 2024-08-20T22:25:20.1684331Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:25:20.1685249Z return self.inference(batched_inputs) 2024-08-20T22:25:20.1686265Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:25:20.1687227Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:25:20.1688469Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:25:20.1689420Z images = ImageList.from_tensors( 2024-08-20T22:25:20.1690432Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:25:20.1691539Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:25:20.1691973Z 2024-08-20T22:25:20.1692223Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:25:20.1692632Z 2024-08-20T22:25:20.1692936Z The above exception was the direct cause of the following exception: 2024-08-20T22:25:20.1693392Z 2024-08-20T22:25:20.1693538Z Traceback (most recent call last): 2024-08-20T22:25:20.1694452Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1999, in get_fake_value 2024-08-20T22:25:20.1695286Z ret_val = wrap_fake_exception( 2024-08-20T22:25:20.1696238Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1484, in wrap_fake_exception 2024-08-20T22:25:20.1697085Z return fn() 2024-08-20T22:25:20.1697857Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2000, in 2024-08-20T22:25:20.1698884Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:25:20.1699900Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:25:20.1700803Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:25:20.1701797Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:25:20.1702602Z return node.target(*args, **kwargs) 2024-08-20T22:25:20.1703676Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:25:20.1704646Z r = self.shape_env.evaluate_expr( 2024-08-20T22:25:20.1705622Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:25:20.1706502Z return retlog(event.run(self)) 2024-08-20T22:25:20.1707442Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:25:20.1708297Z return self.f(*args, **kwargs) 2024-08-20T22:25:20.1709340Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:25:20.1710693Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:25:20.1712167Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:25:20.1713151Z raise self._make_data_dependent_error( 2024-08-20T22:25:20.1714454Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:25:20.1716088Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:25:20.1716752Z 2024-08-20T22:25:20.1717024Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:25:20.1718082Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:25:20.1718899Z return node.target(*args, **kwargs) 2024-08-20T22:25:20.1719192Z 2024-08-20T22:25:20.1719399Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:25:20.1720196Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:25:20.1721192Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:25:20.1722542Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:25:20.1723381Z 2024-08-20T22:25:20.1723533Z User Stack (most recent call last): 2024-08-20T22:25:20.1723953Z (snipped, see stack below for prefix) 2024-08-20T22:25:20.1724981Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:25:20.1725899Z return self.inference(batched_inputs) 2024-08-20T22:25:20.1726925Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:25:20.1727866Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:25:20.1728985Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:25:20.1729936Z images = ImageList.from_tensors( 2024-08-20T22:25:20.1730957Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:25:20.1732074Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:25:20.1732494Z 2024-08-20T22:25:20.1732749Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:25:20.1733172Z 2024-08-20T22:25:20.1733461Z During handling of the above exception, another exception occurred: 2024-08-20T22:25:20.1733923Z 2024-08-20T22:25:20.1734055Z Traceback (most recent call last): 2024-08-20T22:25:20.1734786Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:25:20.1735553Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:25:20.1736359Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:25:20.1737328Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:25:20.1738202Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:25:20.1739029Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:25:20.1740063Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:25:20.1740978Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:25:20.1741936Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:25:20.1742776Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:25:20.1744015Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:25:20.1744943Z return self._call_impl(*args, **kwargs) 2024-08-20T22:25:20.1745925Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:25:20.1746794Z return forward_call(*args, **kwargs) 2024-08-20T22:25:20.1747691Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:25:20.1748478Z return fn(*args, **kwargs) 2024-08-20T22:25:20.1749598Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:25:20.1750531Z return self._call_impl(*args, **kwargs) 2024-08-20T22:25:20.1751495Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:25:20.1752357Z return forward_call(*args, **kwargs) 2024-08-20T22:25:20.1753316Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:25:20.1754193Z return self._torchdynamo_orig_callable( 2024-08-20T22:25:20.1755279Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:25:20.1756181Z return _compile( 2024-08-20T22:25:20.1757016Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:25:20.1757970Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:25:20.1759122Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:25:20.1760014Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:25:20.1760985Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:25:20.1761773Z return function(*args, **kwargs) 2024-08-20T22:25:20.1762675Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:25:20.1763534Z out_code = transform_code_object(code, transform) 2024-08-20T22:25:20.1764628Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:25:20.1765592Z transformations(instructions, code_options) 2024-08-20T22:25:20.1766480Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:25:20.1767217Z return fn(*args, **kwargs) 2024-08-20T22:25:20.1768272Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:25:20.1769079Z tracer.run() 2024-08-20T22:25:20.1769887Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:25:20.1770666Z super().run() 2024-08-20T22:25:20.1771472Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:25:20.1772269Z while self.step(): 2024-08-20T22:25:20.1773100Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:25:20.1773953Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:25:20.1774925Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:25:20.1775749Z return inner_fn(self, inst) 2024-08-20T22:25:20.1776700Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:25:20.1777562Z self.call_function(fn, args, {}) 2024-08-20T22:25:20.1778619Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:25:20.1779959Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:25:20.1781182Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:25:20.1782142Z return super().call_function(tx, args, kwargs) 2024-08-20T22:25:20.1783213Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:25:20.1784178Z return super().call_function(tx, args, kwargs) 2024-08-20T22:25:20.1785331Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:25:20.1786455Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:25:20.1787762Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:25:20.1788951Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:25:20.1790170Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:25:20.1791247Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:25:20.1792300Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:25:20.1793104Z tracer.run() 2024-08-20T22:25:20.1793875Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:25:20.1794631Z while self.step(): 2024-08-20T22:25:20.1795421Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:25:20.1796227Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:25:20.1797156Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:25:20.1797940Z return inner_fn(self, inst) 2024-08-20T22:25:20.1799004Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:25:20.1799868Z self.call_function(fn, args, {}) 2024-08-20T22:25:20.1800840Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:25:20.1801958Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:25:20.1803140Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:25:20.1804063Z return super().call_function(tx, args, kwargs) 2024-08-20T22:25:20.1805106Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:25:20.1806031Z return super().call_function(tx, args, kwargs) 2024-08-20T22:25:20.1807067Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:25:20.1808134Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:25:20.1809413Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:25:20.1810565Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:25:20.1811753Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:25:20.1812679Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:25:20.1813709Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:25:20.1814555Z tracer.run() 2024-08-20T22:25:20.1815355Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:25:20.1816147Z while self.step(): 2024-08-20T22:25:20.1816963Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:25:20.1817826Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:25:20.1818897Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:25:20.1819930Z return inner_fn(self, inst) 2024-08-20T22:25:20.1821001Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:25:20.1821934Z self.call_function(fn, args, kwargs) 2024-08-20T22:25:20.1822953Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:25:20.1824118Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:25:20.1825338Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:25:20.1826284Z return super().call_function(tx, args, kwargs) 2024-08-20T22:25:20.1827479Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:25:20.1828605Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:25:20.1829927Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:25:20.1831218Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:25:20.1832479Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:25:20.1833366Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:25:20.1834365Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:25:20.1835151Z tracer.run() 2024-08-20T22:25:20.1835916Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:25:20.1836677Z while self.step(): 2024-08-20T22:25:20.1837466Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:25:20.1838281Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:25:20.1839199Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:25:20.1839979Z return inner_fn(self, inst) 2024-08-20T22:25:20.1840816Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:25:20.1841692Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:25:20.1842725Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:25:20.1843887Z return handler(tx, args, kwargs) 2024-08-20T22:25:20.1845002Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:25:20.1845990Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:25:20.1846988Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:25:20.1848023Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:25:20.1849197Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:25:20.1850305Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:25:20.1851416Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:25:20.1852260Z raise UserError( # noqa: B904 2024-08-20T22:25:20.1855063Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:25:20.1857594Z 2024-08-20T22:25:20.1857860Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:25:20.1859082Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:25:20.1859914Z return node.target(*args, **kwargs) 2024-08-20T22:25:20.1860209Z 2024-08-20T22:25:20.1860415Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:25:20.1861213Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:25:20.1862208Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:25:20.1863550Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:25:20.1864397Z 2024-08-20T22:25:20.1864551Z User Stack (most recent call last): 2024-08-20T22:25:20.1864976Z (snipped, see stack below for prefix) 2024-08-20T22:25:20.1866006Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:25:20.1866912Z return self.inference(batched_inputs) 2024-08-20T22:25:20.1867944Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:25:20.1868887Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:25:20.1870008Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:25:20.1870966Z images = ImageList.from_tensors( 2024-08-20T22:25:20.1872082Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:25:20.1873175Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:25:20.1873583Z 2024-08-20T22:25:20.1873825Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:25:20.1875005Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:25:20.1875824Z 2024-08-20T22:25:20.1875929Z from user code: 2024-08-20T22:25:20.1876800Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:25:20.1877670Z return self.inference(batched_inputs) 2024-08-20T22:25:20.1878669Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:25:20.1879664Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:25:20.1880710Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:25:20.1881578Z images = ImageList.from_tensors( 2024-08-20T22:25:20.1882525Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:25:20.1883553Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:25:20.1883942Z 2024-08-20T22:25:20.1884229Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:25:20.1884653Z 2024-08-20T22:25:20.1884937Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:25:20.1885447Z fail_to_run 2024-08-20T22:25:20.1886017Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:25:20.1886679Z TIMING: 2024-08-20T22:25:20.1887211Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:48 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:25:20.1888038Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:25:23.2878652Z 2024-08-20T22:25:30.9462560Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:25:30.9463078Z loading model: 0it [00:07, ?it/s] 2024-08-20T22:25:30.9464985Z cpu eval detectron2_fasterrcnn_r_50_fpn 2024-08-20T22:25:45.1621583Z WARNING:common:fp64 golden ref were not generated for detectron2_fasterrcnn_r_50_fpn. Setting accuracy check to cosine 2024-08-20T22:25:45.2031924Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:25:55.8548912Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:06.5155619Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:12.2272234Z W0820 22:26:12.226690 17151 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Eq(u0, 800), hint=None, expect_rational=True, size_oblivious=True, forcing_spec=False 2024-08-20T22:26:12.2274941Z E0820 22:26:12.226856 17151 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Eq(u0, 800), None), **{'fx_node': eq_3, 'size_oblivious': True}) 2024-08-20T22:26:12.2536458Z ERROR:common: 2024-08-20T22:26:12.2536824Z Traceback (most recent call last): 2024-08-20T22:26:12.2537899Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:26:12.2538842Z return node.target(*args, **kwargs) 2024-08-20T22:26:12.2545337Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:26:12.2546370Z r = self.shape_env.evaluate_expr( 2024-08-20T22:26:12.2547392Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:26:12.2548303Z return retlog(event.run(self)) 2024-08-20T22:26:12.2549262Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:26:12.2550124Z return self.f(*args, **kwargs) 2024-08-20T22:26:12.2551210Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:26:12.2552571Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:26:12.2554141Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:26:12.2555145Z raise self._make_data_dependent_error( 2024-08-20T22:26:12.2556513Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:26:12.2557577Z 2024-08-20T22:26:12.2557860Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:26:12.2558924Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:26:12.2559747Z return node.target(*args, **kwargs) 2024-08-20T22:26:12.2560047Z 2024-08-20T22:26:12.2560261Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:26:12.2561071Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:26:12.2562073Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:26:12.2563320Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:26:12.2564168Z 2024-08-20T22:26:12.2564322Z User Stack (most recent call last): 2024-08-20T22:26:12.2564745Z (snipped, see stack below for prefix) 2024-08-20T22:26:12.2565768Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:26:12.2566710Z return self.inference(batched_inputs) 2024-08-20T22:26:12.2567745Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:26:12.2568889Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:26:12.2570031Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:26:12.2570997Z images = ImageList.from_tensors( 2024-08-20T22:26:12.2572034Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:26:12.2573158Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:26:12.2573584Z 2024-08-20T22:26:12.2573841Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:26:12.2574271Z 2024-08-20T22:26:12.2574668Z The above exception was the direct cause of the following exception: 2024-08-20T22:26:12.2575142Z 2024-08-20T22:26:12.2575282Z Traceback (most recent call last): 2024-08-20T22:26:12.2576226Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1999, in get_fake_value 2024-08-20T22:26:12.2577077Z ret_val = wrap_fake_exception( 2024-08-20T22:26:12.2578043Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 1484, in wrap_fake_exception 2024-08-20T22:26:12.2579048Z return fn() 2024-08-20T22:26:12.2579859Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2000, in 2024-08-20T22:26:12.2580775Z lambda: run_node(tx.output, node, args, kwargs, nnmodule) 2024-08-20T22:26:12.2581794Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2135, in run_node 2024-08-20T22:26:12.2582711Z raise RuntimeError(make_error_message(e)).with_traceback( 2024-08-20T22:26:12.2583727Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:26:12.2584543Z return node.target(*args, **kwargs) 2024-08-20T22:26:12.2585614Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 503, in guard_size_oblivious 2024-08-20T22:26:12.2586577Z r = self.shape_env.evaluate_expr( 2024-08-20T22:26:12.2587573Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:26:12.2588471Z return retlog(event.run(self)) 2024-08-20T22:26:12.2589409Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:26:12.2590284Z return self.f(*args, **kwargs) 2024-08-20T22:26:12.2591357Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:26:12.2592713Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:26:12.2594185Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5247, in _evaluate_expr 2024-08-20T22:26:12.2595187Z raise self._make_data_dependent_error( 2024-08-20T22:26:12.2596498Z RuntimeError: Failed running call_function (*(FakeTensor(..., size=(4, 3, u0, u1)), (0, Ellipsis, slice(None, 800, None), slice(None, 1199, None))), **{}): 2024-08-20T22:26:12.2598074Z Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:26:12.2598731Z 2024-08-20T22:26:12.2599024Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:26:12.2600076Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:26:12.2600905Z return node.target(*args, **kwargs) 2024-08-20T22:26:12.2601222Z 2024-08-20T22:26:12.2601420Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:26:12.2602240Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:26:12.2603327Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:26:12.2604591Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:26:12.2605457Z 2024-08-20T22:26:12.2605598Z User Stack (most recent call last): 2024-08-20T22:26:12.2606048Z (snipped, see stack below for prefix) 2024-08-20T22:26:12.2607078Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:26:12.2607985Z return self.inference(batched_inputs) 2024-08-20T22:26:12.2609028Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:26:12.2610067Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:26:12.2611209Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:26:12.2612169Z images = ImageList.from_tensors( 2024-08-20T22:26:12.2613201Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:26:12.2614331Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:26:12.2614762Z 2024-08-20T22:26:12.2615037Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:26:12.2615456Z 2024-08-20T22:26:12.2615768Z During handling of the above exception, another exception occurred: 2024-08-20T22:26:12.2616226Z 2024-08-20T22:26:12.2616370Z Traceback (most recent call last): 2024-08-20T22:26:12.2617116Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:26:12.2617911Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:26:12.2618816Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:26:12.2619867Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:26:12.2620758Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:26:12.2621517Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:26:12.2622569Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:26:12.2623471Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:26:12.2624430Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:26:12.2625272Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:26:12.2626310Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:26:12.2627232Z return self._call_impl(*args, **kwargs) 2024-08-20T22:26:12.2628219Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:26:12.2629090Z return forward_call(*args, **kwargs) 2024-08-20T22:26:12.2630014Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:26:12.2630805Z return fn(*args, **kwargs) 2024-08-20T22:26:12.2631885Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:26:12.2632799Z return self._call_impl(*args, **kwargs) 2024-08-20T22:26:12.2633757Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:26:12.2634605Z return forward_call(*args, **kwargs) 2024-08-20T22:26:12.2635534Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:26:12.2636395Z return self._torchdynamo_orig_callable( 2024-08-20T22:26:12.2637422Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:26:12.2638243Z return _compile( 2024-08-20T22:26:12.2639145Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:26:12.2640059Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:26:12.2641112Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:26:12.2642021Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:26:12.2642996Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:26:12.2644203Z return function(*args, **kwargs) 2024-08-20T22:26:12.2645199Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:26:12.2646162Z out_code = transform_code_object(code, transform) 2024-08-20T22:26:12.2647342Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:26:12.2648392Z transformations(instructions, code_options) 2024-08-20T22:26:12.2649364Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:26:12.2650176Z return fn(*args, **kwargs) 2024-08-20T22:26:12.2651083Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:26:12.2651938Z tracer.run() 2024-08-20T22:26:12.2652767Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:26:12.2653598Z super().run() 2024-08-20T22:26:12.2654431Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:26:12.2655253Z while self.step(): 2024-08-20T22:26:12.2656114Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:26:12.2657063Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:26:12.2658003Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:26:12.2658872Z return inner_fn(self, inst) 2024-08-20T22:26:12.2660007Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:26:12.2660919Z self.call_function(fn, args, {}) 2024-08-20T22:26:12.2661921Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:26:12.2663090Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:26:12.2664312Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:26:12.2665274Z return super().call_function(tx, args, kwargs) 2024-08-20T22:26:12.2666358Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:26:12.2667326Z return super().call_function(tx, args, kwargs) 2024-08-20T22:26:12.2668413Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:26:12.2669521Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:26:12.2670843Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:26:12.2672092Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:26:12.2673227Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:26:12.2674250Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:26:12.2675257Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:26:12.2676284Z tracer.run() 2024-08-20T22:26:12.2677099Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:26:12.2677910Z while self.step(): 2024-08-20T22:26:12.2678742Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:26:12.2679617Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:26:12.2680603Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:26:12.2681514Z return inner_fn(self, inst) 2024-08-20T22:26:12.2682456Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:26:12.2683348Z self.call_function(fn, args, {}) 2024-08-20T22:26:12.2684321Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:26:12.2685453Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:26:12.2686632Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:26:12.2687581Z return super().call_function(tx, args, kwargs) 2024-08-20T22:26:12.2688638Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:26:12.2689588Z return super().call_function(tx, args, kwargs) 2024-08-20T22:26:12.2690622Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:26:12.2691821Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:26:12.2693045Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:26:12.2694147Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:26:12.2695289Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:26:12.2696175Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:26:12.2697181Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:26:12.2698003Z tracer.run() 2024-08-20T22:26:12.2698861Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:26:12.2699914Z while self.step(): 2024-08-20T22:26:12.2700798Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:26:12.2701695Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:26:12.2702709Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:26:12.2703564Z return inner_fn(self, inst) 2024-08-20T22:26:12.2704571Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:26:12.2705506Z self.call_function(fn, args, kwargs) 2024-08-20T22:26:12.2706521Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:26:12.2707688Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:26:12.2708901Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:26:12.2709943Z return super().call_function(tx, args, kwargs) 2024-08-20T22:26:12.2711037Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:26:12.2712337Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:26:12.2713543Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:26:12.2714652Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:26:12.2715785Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:26:12.2717757Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:26:12.2718753Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:26:12.2719567Z tracer.run() 2024-08-20T22:26:12.2720344Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:26:12.2721108Z while self.step(): 2024-08-20T22:26:12.2721907Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:26:12.2722718Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:26:12.2723655Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:26:12.2724451Z return inner_fn(self, inst) 2024-08-20T22:26:12.2725297Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 297, in impl 2024-08-20T22:26:12.2726182Z self.push(fn_var.call_function(self, self.popn(nargs), {})) 2024-08-20T22:26:12.2727243Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 982, in call_function 2024-08-20T22:26:12.2728086Z return handler(tx, args, kwargs) 2024-08-20T22:26:12.2729104Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builtin.py", line 958, in _handle_insert_op_in_graph 2024-08-20T22:26:12.2730017Z return wrap_fx_proxy(tx, proxy) 2024-08-20T22:26:12.2730959Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 1946, in wrap_fx_proxy 2024-08-20T22:26:12.2731922Z return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs) 2024-08-20T22:26:12.2733060Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/builder.py", line 2033, in wrap_fx_proxy_cls 2024-08-20T22:26:12.2734103Z example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True) 2024-08-20T22:26:12.2735140Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2054, in get_fake_value 2024-08-20T22:26:12.2735937Z raise UserError( # noqa: B904 2024-08-20T22:26:12.2738533Z torch._dynamo.exc.UserError: Tried to use data-dependent value in the subsequent computation. This can happen when we encounter unbounded dynamic value that is unknown during tracing time. You will need to explicitly give hint to the compiler. Please take a look at torch._check OR torch._check_is_size APIs. Could not guard on data-dependent expression Eq(u0, 800) (unhinted: Eq(u0, 800)). (Size-like symbols: u0) 2024-08-20T22:26:12.2741157Z 2024-08-20T22:26:12.2741443Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:26:12.2742519Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/utils.py", line 2117, in run_node 2024-08-20T22:26:12.2743507Z return node.target(*args, **kwargs) 2024-08-20T22:26:12.2743815Z 2024-08-20T22:26:12.2744015Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:26:12.2744838Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:26:12.2745979Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:26:12.2747241Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:26:12.2748094Z 2024-08-20T22:26:12.2748245Z User Stack (most recent call last): 2024-08-20T22:26:12.2748677Z (snipped, see stack below for prefix) 2024-08-20T22:26:12.2749723Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:26:12.2750647Z return self.inference(batched_inputs) 2024-08-20T22:26:12.2751683Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:26:12.2752712Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:26:12.2753857Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:26:12.2754825Z images = ImageList.from_tensors( 2024-08-20T22:26:12.2756057Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:26:12.2757086Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:26:12.2757496Z 2024-08-20T22:26:12.2757735Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:26:12.2758864Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:26:12.2759638Z 2024-08-20T22:26:12.2759752Z from user code: 2024-08-20T22:26:12.2760591Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward 2024-08-20T22:26:12.2761433Z return self.inference(batched_inputs) 2024-08-20T22:26:12.2762396Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference 2024-08-20T22:26:12.2763288Z images = self.preprocess_image(batched_inputs) 2024-08-20T22:26:12.2764334Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 229, in preprocess_image 2024-08-20T22:26:12.2765217Z images = ImageList.from_tensors( 2024-08-20T22:26:12.2766167Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/detectron2/structures/image_list.py", line 127, in from_tensors 2024-08-20T22:26:12.2767208Z batched_imgs[i, ..., : img.shape[-2], : img.shape[-1]].copy_(img) 2024-08-20T22:26:12.2767604Z 2024-08-20T22:26:12.2767896Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:26:12.2768324Z 2024-08-20T22:26:12.2768623Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:26:12.2769154Z fail_to_run 2024-08-20T22:26:12.2769732Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:26:12.2770410Z TIMING: 2024-08-20T22:26:12.2770962Z STATS: call_* op count: 0 | FakeTensorMode.__torch_dispatch__:51 | FakeTensor.__torch_dispatch__:1 2024-08-20T22:26:12.2771796Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:26:15.3443627Z 2024-08-20T22:26:34.6137799Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:34.6138412Z loading model: 0it [00:19, ?it/s] 2024-08-20T22:26:34.6141837Z Traceback (most recent call last): 2024-08-20T22:26:34.6142418Z File "benchmarks/dynamo/torchbench.py", line 460, in 2024-08-20T22:26:34.6142983Z torchbench_main() 2024-08-20T22:26:34.6143653Z File "benchmarks/dynamo/torchbench.py", line 456, in torchbench_main 2024-08-20T22:26:34.6144319Z main(TorchBenchmarkRunner(), original_dir) 2024-08-20T22:26:34.6145076Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4087, in main 2024-08-20T22:26:34.6145843Z process_entry(0, runner, original_dir, args) 2024-08-20T22:26:34.6146898Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4019, in process_entry 2024-08-20T22:26:34.6148955Z return run(runner, args, original_dir) 2024-08-20T22:26:34.6149661Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4661, in run 2024-08-20T22:26:34.6154183Z assert marked, f"nothing in example_inputs had a dim with {batch_size}" 2024-08-20T22:26:34.6154929Z AssertionError: nothing in example_inputs had a dim with 4 2024-08-20T22:26:35.1786000Z Run failed with return code: 1 2024-08-20T22:26:35.1786494Z Output: None 2024-08-20T22:26:35.1786805Z Error: None 2024-08-20T22:26:37.6637258Z 2024-08-20T22:26:47.8490337Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:47.8490825Z loading model: 0it [00:10, ?it/s] 2024-08-20T22:26:47.8491575Z Traceback (most recent call last): 2024-08-20T22:26:47.8492120Z File "benchmarks/dynamo/torchbench.py", line 460, in 2024-08-20T22:26:47.8492671Z torchbench_main() 2024-08-20T22:26:47.8493170Z File "benchmarks/dynamo/torchbench.py", line 456, in torchbench_main 2024-08-20T22:26:47.8507912Z main(TorchBenchmarkRunner(), original_dir) 2024-08-20T22:26:47.8508672Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4087, in main 2024-08-20T22:26:47.8509423Z process_entry(0, runner, original_dir, args) 2024-08-20T22:26:47.8510211Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4019, in process_entry 2024-08-20T22:26:47.8510983Z return run(runner, args, original_dir) 2024-08-20T22:26:47.8511680Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4661, in run 2024-08-20T22:26:47.8512537Z assert marked, f"nothing in example_inputs had a dim with {batch_size}" 2024-08-20T22:26:47.8513257Z AssertionError: nothing in example_inputs had a dim with 4 2024-08-20T22:26:48.4330490Z Run failed with return code: 1 2024-08-20T22:26:48.4330993Z Output: None 2024-08-20T22:26:48.4331281Z Error: None 2024-08-20T22:26:51.0357975Z 2024-08-20T22:27:09.2056674Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:27:09.2057207Z loading model: 0it [00:18, ?it/s] 2024-08-20T22:27:09.2057884Z Traceback (most recent call last): 2024-08-20T22:27:09.2058548Z File "benchmarks/dynamo/torchbench.py", line 460, in 2024-08-20T22:27:09.2059330Z torchbench_main() 2024-08-20T22:27:09.2059861Z File "benchmarks/dynamo/torchbench.py", line 456, in torchbench_main 2024-08-20T22:27:09.2060530Z main(TorchBenchmarkRunner(), original_dir) 2024-08-20T22:27:09.2061265Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4087, in main 2024-08-20T22:27:09.2064331Z process_entry(0, runner, original_dir, args) 2024-08-20T22:27:09.2065141Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4019, in process_entry 2024-08-20T22:27:09.2068806Z return run(runner, args, original_dir) 2024-08-20T22:27:09.2069532Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4661, in run 2024-08-20T22:27:09.2074311Z assert marked, f"nothing in example_inputs had a dim with {batch_size}" 2024-08-20T22:27:09.2075147Z AssertionError: nothing in example_inputs had a dim with 4 2024-08-20T22:27:09.8044919Z Run failed with return code: 1 2024-08-20T22:27:09.8046140Z Output: None 2024-08-20T22:27:09.8048411Z Error: None 2024-08-20T22:27:12.3284271Z 2024-08-20T22:27:20.6408359Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:27:20.6408845Z loading model: 0it [00:08, ?it/s] 2024-08-20T22:27:20.6409261Z Traceback (most recent call last): 2024-08-20T22:27:20.6409817Z File "benchmarks/dynamo/torchbench.py", line 460, in 2024-08-20T22:27:20.6411545Z torchbench_main() 2024-08-20T22:27:20.6412132Z File "benchmarks/dynamo/torchbench.py", line 456, in torchbench_main 2024-08-20T22:27:20.6412800Z main(TorchBenchmarkRunner(), original_dir) 2024-08-20T22:27:20.6413546Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4087, in main 2024-08-20T22:27:20.6415615Z process_entry(0, runner, original_dir, args) 2024-08-20T22:27:20.6416702Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4019, in process_entry 2024-08-20T22:27:20.6420502Z return run(runner, args, original_dir) 2024-08-20T22:27:20.6421198Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4661, in run 2024-08-20T22:27:20.6425858Z assert marked, f"nothing in example_inputs had a dim with {batch_size}" 2024-08-20T22:27:20.6426584Z AssertionError: nothing in example_inputs had a dim with 4 2024-08-20T22:27:21.2054601Z Run failed with return code: 1 2024-08-20T22:27:21.2055093Z Output: None 2024-08-20T22:27:21.2055388Z Error: None 2024-08-20T22:27:23.6873474Z 2024-08-20T22:27:32.0281321Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:27:32.0282604Z loading model: 0it [00:08, ?it/s] 2024-08-20T22:27:32.0283318Z cpu eval dlrm 2024-08-20T22:27:32.6843729Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:32.9106147Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:33.1335221Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:58.8719866Z Error: weights_offset must be aligned to 16K boundary 2024-08-20T22:27:58.8723576Z ERROR:common: 2024-08-20T22:27:58.8723881Z Traceback (most recent call last): 2024-08-20T22:27:58.8727460Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:27:58.8728259Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:27:58.8729082Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:27:58.8730084Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:27:58.8730945Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1380, in load 2024-08-20T22:27:58.8731740Z cls.cache[key] = torch._export.aot_load(so_path, device) 2024-08-20T22:27:58.8732932Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_export/__init__.py", line 300, in aot_load 2024-08-20T22:27:58.8734130Z runner = torch._C._aoti.AOTIModelContainerRunnerCpu(so_path, 1) # type: ignore[call-arg] 2024-08-20T22:27:58.8735945Z RuntimeError: create_func_( &container_handle_, num_models, device_str.c_str(), cubin_dir.empty() ? nullptr : cubin_dir.c_str()) API call failed at /var/lib/jenkins/workspace/torch/csrc/inductor/aoti_runner/model_container_runner.cpp, line 70 2024-08-20T22:27:58.8737619Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:27:58.8738182Z fail_to_run 2024-08-20T22:27:58.8738924Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:27:58.8739904Z TIMING: backend_compile:0.0001 entire_frame_compile:0.29909 total_wall_time:0.29909 2024-08-20T22:27:58.8741126Z STATS: call_* op count: 36 | FakeTensor.__torch_dispatch__:213 | FakeTensorMode.__torch_dispatch__:2671 | ProxyTorchDispatchMode.__torch_dispatch__:662 2024-08-20T22:27:58.8742281Z Dynamo produced 1 graphs covering 36 ops with 0 graph breaks (0 unique) 2024-08-20T22:28:02.2705872Z 2024-08-20T22:28:03.5005679Z loading model: 0it [00:00, ?it/s]Downloading https://doctr-static.mindee.com/models?id=v0.7.0/db_resnet50-79bd7d70.pt&src=0 to /var/lib/jenkins/.cache/doctr/models/db_resnet50-79bd7d70.pt 2024-08-20T22:28:03.8570409Z 2024-08-20T22:28:03.8570419Z 2024-08-20T22:28:03.9572720Z 0% 0/102021912 [00:00 2024-08-20T22:28:43.1812198Z [ 2024-08-20T22:28:43.1813016Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/detection/core.py", line 92, in 2024-08-20T22:28:43.1814038Z cv2.morphologyEx(bmap[..., idx], cv2.MORPH_OPEN, self._opening_kernel) 2024-08-20T22:28:43.1814501Z 2024-08-20T22:28:43.1814796Z Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information 2024-08-20T22:28:43.1815268Z 2024-08-20T22:28:43.1815565Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:28:43.1816151Z fail_to_run 2024-08-20T22:28:43.1816766Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:28:43.1817475Z TIMING: 2024-08-20T22:28:43.1818142Z STATS: call_* op count: 0 | FakeTensor.__torch_dispatch__:605 | FakeTensorMode.__torch_dispatch__:4086 2024-08-20T22:28:43.1819155Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:28:46.3084365Z 2024-08-20T22:28:49.6807766Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:28:49.6808386Z loading model: 0it [00:03, ?it/s] 2024-08-20T22:28:49.6808961Z cpu eval doctr_reco_predictor 2024-08-20T22:28:49.9056901Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:28:49.9813486Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:28:50.0532775Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:08.5482868Z W0820 22:29:08.547388 19385 torch/fx/experimental/symbolic_shapes.py:5131] [0/0] failed during evaluate_expr(Ne(u0, 123), hint=None, expect_rational=True, size_oblivious=False, forcing_spec=False 2024-08-20T22:29:08.5485152Z E0820 22:29:08.547537 19385 torch/fx/experimental/recording.py:298] [0/0] failed while running evaluate_expr(*(Ne(u0, 123), None), **{'fx_node': ne_210}) 2024-08-20T22:29:08.5542350Z ERROR:common: 2024-08-20T22:29:08.5542696Z Traceback (most recent call last): 2024-08-20T22:29:08.5545593Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/tensor.py", line 1103, in evaluate_expr 2024-08-20T22:29:08.5546558Z return guard_scalar(self.sym_num) 2024-08-20T22:29:08.5547655Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 845, in guard_scalar 2024-08-20T22:29:08.5548613Z return guard_bool(a) 2024-08-20T22:29:08.5549606Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 1018, in guard_bool 2024-08-20T22:29:08.5550706Z return a.node.guard_bool("", 0) # NB: uses Python backtrace 2024-08-20T22:29:08.5551847Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/sym_node.py", line 451, in guard_bool 2024-08-20T22:29:08.5552954Z r = self.shape_env.evaluate_expr(self.expr, self.hint, fx_node=self.fx_node) 2024-08-20T22:29:08.5554166Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 292, in wrapper 2024-08-20T22:29:08.5555043Z return retlog(event.run(self)) 2024-08-20T22:29:08.5556017Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/recording.py", line 165, in run 2024-08-20T22:29:08.5556881Z return self.f(*args, **kwargs) 2024-08-20T22:29:08.5557948Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5129, in evaluate_expr 2024-08-20T22:29:08.5559307Z return self._evaluate_expr(orig_expr, hint, fx_node, expect_rational, size_oblivious, forcing_spec=forcing_spec) 2024-08-20T22:29:08.5560767Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5168, in _evaluate_expr 2024-08-20T22:29:08.5561761Z concrete_val = compute_concrete_val() 2024-08-20T22:29:08.5562903Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 5149, in compute_concrete_val 2024-08-20T22:29:08.5564092Z return self.size_hint(orig_expr) 2024-08-20T22:29:08.5565631Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 4671, in size_hint 2024-08-20T22:29:08.5566685Z raise self._make_data_dependent_error(result_expr, expr) 2024-08-20T22:29:08.5568165Z torch.fx.experimental.symbolic_shapes.GuardOnDataDependentSymNode: Could not guard on data-dependent expression Ne(u0, 123) (unhinted: Ne(u0, 123)). (Size-like symbols: none) 2024-08-20T22:29:08.5569239Z 2024-08-20T22:29:08.5569527Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:29:08.5570697Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/tensor.py", line 1103, in evaluate_expr 2024-08-20T22:29:08.5571821Z return guard_scalar(self.sym_num) 2024-08-20T22:29:08.5572124Z 2024-08-20T22:29:08.5572317Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:29:08.5573124Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:29:08.5574129Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:29:08.5575361Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:29:08.5576216Z 2024-08-20T22:29:08.5576353Z User Stack (most recent call last): 2024-08-20T22:29:08.5576882Z (snipped, see stack below for prefix) 2024-08-20T22:29:08.5577948Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 224, in forward 2024-08-20T22:29:08.5579115Z out["preds"] = self.postprocessor(logits) 2024-08-20T22:29:08.5580205Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 99, in __call__ 2024-08-20T22:29:08.5581363Z return self.ctc_best_path(logits=logits, vocab=self.vocab, blank=len(self.vocab)) 2024-08-20T22:29:08.5582666Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 78, in ctc_best_path 2024-08-20T22:29:08.5583598Z words = [ 2024-08-20T22:29:08.5584524Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-08-20T22:29:08.5585651Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-08-20T22:29:08.5586882Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-08-20T22:29:08.5588005Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-08-20T22:29:08.5588494Z 2024-08-20T22:29:08.5588749Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:29:08.5589161Z 2024-08-20T22:29:08.5589461Z During handling of the above exception, another exception occurred: 2024-08-20T22:29:08.5589913Z 2024-08-20T22:29:08.5590060Z Traceback (most recent call last): 2024-08-20T22:29:08.5590995Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 622, in transform 2024-08-20T22:29:08.5591822Z tracer.run() 2024-08-20T22:29:08.5592658Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2780, in run 2024-08-20T22:29:08.5593476Z super().run() 2024-08-20T22:29:08.5594289Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:29:08.5595114Z while self.step(): 2024-08-20T22:29:08.5595965Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:29:08.5596853Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:29:08.5597843Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:29:08.5598691Z return inner_fn(self, inst) 2024-08-20T22:29:08.5599667Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:29:08.5600568Z self.call_function(fn, args, {}) 2024-08-20T22:29:08.5601544Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:29:08.5602704Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:29:08.5603948Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/user_defined.py", line 876, in call_function 2024-08-20T22:29:08.5604951Z return self.call_method(tx, "__call__", args, kwargs) 2024-08-20T22:29:08.5606148Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/user_defined.py", line 736, in call_method 2024-08-20T22:29:08.5607233Z return UserMethodVariable(method, self, source=source).call_function( 2024-08-20T22:29:08.5608446Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 383, in call_function 2024-08-20T22:29:08.5609407Z return super().call_function(tx, args, kwargs) 2024-08-20T22:29:08.5610486Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:29:08.5611427Z return super().call_function(tx, args, kwargs) 2024-08-20T22:29:08.5612498Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:29:08.5613693Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:29:08.5615010Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:29:08.5616199Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:29:08.5617398Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:29:08.5618443Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:29:08.5619529Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:29:08.5620397Z tracer.run() 2024-08-20T22:29:08.5621207Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:29:08.5622031Z while self.step(): 2024-08-20T22:29:08.5622879Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:29:08.5623764Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:29:08.5624755Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:29:08.5625604Z return inner_fn(self, inst) 2024-08-20T22:29:08.5626599Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1676, in CALL_FUNCTION_KW 2024-08-20T22:29:08.5627526Z self.call_function(fn, args, kwargs) 2024-08-20T22:29:08.5628517Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:29:08.5629681Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:29:08.5630912Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 322, in call_function 2024-08-20T22:29:08.5631980Z return super().call_function(tx, args, kwargs) 2024-08-20T22:29:08.5633041Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:29:08.5634120Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:29:08.5635393Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:29:08.5636549Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:29:08.5637733Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:29:08.5638658Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:29:08.5639706Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:29:08.5640556Z tracer.run() 2024-08-20T22:29:08.5641359Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:29:08.5642233Z while self.step(): 2024-08-20T22:29:08.5643058Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:29:08.5644563Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:29:08.5645593Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:29:08.5646453Z return inner_fn(self, inst) 2024-08-20T22:29:08.5647412Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:29:08.5648314Z self.call_function(fn, args, {}) 2024-08-20T22:29:08.5649453Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:29:08.5650610Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:29:08.5651823Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:29:08.5652942Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:29:08.5654254Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:29:08.5655456Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:29:08.5656754Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:29:08.5657627Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:29:08.5658866Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:29:08.5659737Z tracer.run() 2024-08-20T22:29:08.5660571Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:29:08.5661379Z while self.step(): 2024-08-20T22:29:08.5662239Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:29:08.5663132Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:29:08.5664140Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 578, in wrapper 2024-08-20T22:29:08.5664975Z return inner_fn(self, inst) 2024-08-20T22:29:08.5665948Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 1586, in CALL_FUNCTION 2024-08-20T22:29:08.5666843Z self.call_function(fn, args, {}) 2024-08-20T22:29:08.5667845Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 826, in call_function 2024-08-20T22:29:08.5668987Z self.push(fn.call_function(self, args, kwargs)) # type: ignore[arg-type] 2024-08-20T22:29:08.5670211Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/functions.py", line 106, in call_function 2024-08-20T22:29:08.5671418Z return tx.inline_user_function_return(self, [*self.self_args(), *args], kwargs) 2024-08-20T22:29:08.5672637Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 832, in inline_user_function_return 2024-08-20T22:29:08.5673726Z return InliningInstructionTranslator.inline_call(self, fn, args, kwargs) 2024-08-20T22:29:08.5674837Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 2995, in inline_call 2024-08-20T22:29:08.5675728Z return cls.inline_call_(parent, func, args, kwargs) 2024-08-20T22:29:08.5676730Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 3123, in inline_call_ 2024-08-20T22:29:08.5677531Z tracer.run() 2024-08-20T22:29:08.5678289Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 979, in run 2024-08-20T22:29:08.5679151Z while self.step(): 2024-08-20T22:29:08.5679955Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 891, in step 2024-08-20T22:29:08.5680776Z self.dispatch_table[inst.opcode](self, inst) 2024-08-20T22:29:08.5681684Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/symbolic_convert.py", line 532, in inner 2024-08-20T22:29:08.5682518Z eval_result = value.evaluate_expr(self.output) 2024-08-20T22:29:08.5683507Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/tensor.py", line 1105, in evaluate_expr 2024-08-20T22:29:08.5684340Z raise UserError( # noqa: B904 2024-08-20T22:29:08.5685686Z torch._dynamo.exc.UserError: Consider annotating your code using torch._check*(). Could not guard on data-dependent expression Ne(u0, 123) (unhinted: Ne(u0, 123)). (Size-like symbols: none) 2024-08-20T22:29:08.5686707Z 2024-08-20T22:29:08.5686965Z Potential framework code culprit (scroll up for full backtrace): 2024-08-20T22:29:08.5688045Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/variables/tensor.py", line 1103, in evaluate_expr 2024-08-20T22:29:08.5688886Z return guard_scalar(self.sym_num) 2024-08-20T22:29:08.5689149Z 2024-08-20T22:29:08.5689341Z For more information, run with TORCH_LOGS="dynamic" 2024-08-20T22:29:08.5690076Z For extended logs when we create symbols, also add TORCHDYNAMO_EXTENDED_DEBUG_CREATE_SYMBOL="u0" 2024-08-20T22:29:08.5690997Z If you suspect the guard was triggered from C++, add TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:29:08.5692148Z For more debugging help, see https://docs.google.com/document/d/1HSuTTVvYH1pTew89Rtpeu84Ht3nQEFTYhAX3Ypa_xJs/edit?usp=sharing 2024-08-20T22:29:08.5692928Z 2024-08-20T22:29:08.5693067Z User Stack (most recent call last): 2024-08-20T22:29:08.5693457Z (snipped, see stack below for prefix) 2024-08-20T22:29:08.5694438Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 224, in forward 2024-08-20T22:29:08.5695332Z out["preds"] = self.postprocessor(logits) 2024-08-20T22:29:08.5696326Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 99, in __call__ 2024-08-20T22:29:08.5697390Z return self.ctc_best_path(logits=logits, vocab=self.vocab, blank=len(self.vocab)) 2024-08-20T22:29:08.5698863Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 78, in ctc_best_path 2024-08-20T22:29:08.5699796Z words = [ 2024-08-20T22:29:08.5700741Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-08-20T22:29:08.5701878Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-08-20T22:29:08.5703108Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/doctr/models/recognition/crnn/pytorch.py", line 79, in 2024-08-20T22:29:08.5704239Z decode_sequence([k for k, _ in groupby(seq.tolist()) if k != blank], vocab) 2024-08-20T22:29:08.5704730Z 2024-08-20T22:29:08.5704985Z For C++ stack trace, run with TORCHDYNAMO_EXTENDED_DEBUG_CPP=1 2024-08-20T22:29:08.5706195Z For more information about this error, see: https://pytorch.org/docs/main/generated/exportdb/index.html#constrain-as-size-example 2024-08-20T22:29:08.5707030Z 2024-08-20T22:29:08.5707336Z During handling of the above exception, another exception occurred: 2024-08-20T22:29:08.5707788Z 2024-08-20T22:29:08.5707923Z Traceback (most recent call last): 2024-08-20T22:29:08.5708887Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:29:08.5709799Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:29:08.5710759Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1483, in inner 2024-08-20T22:29:08.5711657Z raise constraint_violation_error 2024-08-20T22:29:08.5712619Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1437, in inner 2024-08-20T22:29:08.5713439Z result_traced = opt_f(*args, **kwargs) 2024-08-20T22:29:08.5714535Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:29:08.5715380Z return self._call_impl(*args, **kwargs) 2024-08-20T22:29:08.5716287Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:29:08.5717093Z return forward_call(*args, **kwargs) 2024-08-20T22:29:08.5717926Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 469, in _fn 2024-08-20T22:29:08.5718717Z return fn(*args, **kwargs) 2024-08-20T22:29:08.5719603Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:29:08.5720456Z return self._call_impl(*args, **kwargs) 2024-08-20T22:29:08.5721547Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:29:08.5722392Z return forward_call(*args, **kwargs) 2024-08-20T22:29:08.5723318Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 1238, in __call__ 2024-08-20T22:29:08.5724166Z return self._torchdynamo_orig_callable( 2024-08-20T22:29:08.5725110Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 514, in __call__ 2024-08-20T22:29:08.5725912Z return _compile( 2024-08-20T22:29:08.5726726Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 902, in _compile 2024-08-20T22:29:08.5727682Z guarded_code = compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:29:08.5728778Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 653, in compile_inner 2024-08-20T22:29:08.5729724Z return _compile_inner(code, one_graph, hooks, transform) 2024-08-20T22:29:08.5730741Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_utils_internal.py", line 87, in wrapper_function 2024-08-20T22:29:08.5731570Z return function(*args, **kwargs) 2024-08-20T22:29:08.5732523Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 686, in _compile_inner 2024-08-20T22:29:08.5733437Z out_code = transform_code_object(code, transform) 2024-08-20T22:29:08.5734645Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/bytecode_transformation.py", line 1322, in transform_code_object 2024-08-20T22:29:08.5735609Z transformations(instructions, code_options) 2024-08-20T22:29:08.5736504Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 208, in _fn 2024-08-20T22:29:08.5737258Z return fn(*args, **kwargs) 2024-08-20T22:29:08.5738110Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/convert_frame.py", line 630, in transform 2024-08-20T22:29:08.5739176Z bisect(tracer.output.shape_env) 2024-08-20T22:29:08.5740165Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/validator.py", line 723, in bisect 2024-08-20T22:29:08.5741284Z last_exception = check_shapeenv_fails(shape_env, shape_env._snapshot_tracked_fakes()) 2024-08-20T22:29:08.5742622Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/validator.py", line 705, in check_shapeenv_fails 2024-08-20T22:29:08.5743887Z shape_env.produce_guards( 2024-08-20T22:29:08.5745011Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/fx/experimental/symbolic_shapes.py", line 4185, in produce_guards 2024-08-20T22:29:08.5746000Z raise ConstraintViolationError( 2024-08-20T22:29:08.5747379Z torch.fx.experimental.symbolic_shapes.ConstraintViolationError: Constraints violated (L['x'].size()[0])! For more information, run with TORCH_LOGS="+dynamic". 2024-08-20T22:29:08.5749074Z - Not all values of RelaxedUnspecConstraint(L['x'].size()[0]) are valid because L['x'].size()[0] was inferred to be a constant (4). 2024-08-20T22:29:08.5749833Z 2024-08-20T22:29:08.5749838Z 2024-08-20T22:29:08.5750126Z During handling of the above exception, another exception occurred: 2024-08-20T22:29:08.5750593Z 2024-08-20T22:29:08.5750726Z Traceback (most recent call last): 2024-08-20T22:29:08.5751463Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:29:08.5752245Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:29:08.5753127Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:29:08.5754104Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:29:08.5754982Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:29:08.5755814Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:29:08.5756763Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 569, in _export_to_torch_ir 2024-08-20T22:29:08.5757757Z raise UserError(UserErrorType.CONSTRAINT_VIOLATION, str(e)) # noqa: B904 2024-08-20T22:29:08.5758849Z torch._dynamo.exc.UserError: Constraints violated (L['x'].size()[0])! For more information, run with TORCH_LOGS="+dynamic". 2024-08-20T22:29:08.5760186Z - Not all values of RelaxedUnspecConstraint(L['x'].size()[0]) are valid because L['x'].size()[0] was inferred to be a constant (4). 2024-08-20T22:29:08.5760884Z 2024-08-20T22:29:08.5761173Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:29:08.5761689Z fail_to_run 2024-08-20T22:29:08.5762260Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:08.5762931Z TIMING: 2024-08-20T22:29:08.5763687Z STATS: call_* op count: 0 | FakeTensor.__torch_dispatch__:260 | FakeTensorMode.__torch_dispatch__:19061 | attempt fast:957 | fast is_contiguous:957 2024-08-20T22:29:08.5764723Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:29:12.2276365Z 2024-08-20T22:29:12.9903850Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:12.9904484Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:12.9905045Z cpu eval drq 2024-08-20T22:29:13.0113417Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:13.0194266Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:13.0270688Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:13.4407108Z ERROR:common: 2024-08-20T22:29:13.4407508Z Traceback (most recent call last): 2024-08-20T22:29:13.4408314Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2808, in check_accuracy 2024-08-20T22:29:13.4409108Z optimized_model_iter_fn = optimize_ctx( 2024-08-20T22:29:13.4409931Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1400, in export_aot_inductor 2024-08-20T22:29:13.4410911Z optimized = AOTInductorModelCache.load(model, example_inputs, device) 2024-08-20T22:29:13.4411778Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 1370, in load 2024-08-20T22:29:13.4412520Z gm = torch.export._trace._export_to_torch_ir( 2024-08-20T22:29:13.4413734Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/export/_trace.py", line 552, in _export_to_torch_ir 2024-08-20T22:29:13.4414655Z gm_torch_level, _ = torch._dynamo.export( 2024-08-20T22:29:13.4415597Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1574, in inner 2024-08-20T22:29:13.4416413Z graph = rewrite_signature( 2024-08-20T22:29:13.4417593Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1077, in rewrite_signature 2024-08-20T22:29:13.4418843Z check_user_input_output(flat_results_traced, UserErrorType.INVALID_OUTPUT) 2024-08-20T22:29:13.4420110Z File "/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/_dynamo/eval_frame.py", line 1063, in check_user_input_output 2024-08-20T22:29:13.4421001Z raise UserError( 2024-08-20T22:29:13.4422296Z torch._dynamo.exc.UserError: It looks like one of the outputs with type `` is not supported or pytree-flattenable. 2024-08-20T22:29:13.4426082Z Exported graphs outputs can only contain the following supported types: [, , , , , , , , , , , , , , , , , , ]. 2024-08-20T22:29:13.4429604Z If you are using a custom class object, please register a pytree_flatten/unflatten function using `torch.utils._pytree.register_pytree_node` or `torch.export.register_dataclass`. 2024-08-20T22:29:13.4431074Z TorchDynamo optimized model failed to run because of following error 2024-08-20T22:29:13.4431633Z fail_to_run 2024-08-20T22:29:13.4432222Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:13.4433261Z TIMING: backend_compile:8e-05 entire_frame_compile:0.25436 total_wall_time:0.25436 2024-08-20T22:29:13.4434200Z STATS: call_* op count: 29 | FakeTensor.__torch_dispatch__:66 | FakeTensorMode.__torch_dispatch__:349 2024-08-20T22:29:13.4435099Z Dynamo produced 1 graphs covering 29 ops with 0 graph breaks (0 unique) 2024-08-20T22:29:16.4138392Z 2024-08-20T22:29:16.8104366Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:16.8104992Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:16.8105571Z cpu eval functorch_dp_cifar10 2024-08-20T22:29:16.8522971Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:16.8771519Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:29:16.9002938Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:00.7249036Z /tmp/tmp3ydr6m14/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cles4fqzh5bjmrfn66gfzlwx2uzzdoyzhal7u7ivuvnc5cqhpsuw.cpp: In function ‘void cpp_fused_add_native_group_norm_relu_8(bfloat16*, const bfloat16*, const bfloat16*, const float*, const float*, const float*, const float*, const float*, const float*, float*, float*, long int)’: 2024-08-20T22:30:00.7257083Z /tmp/tmp3ydr6m14/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cles4fqzh5bjmrfn66gfzlwx2uzzdoyzhal7u7ivuvnc5cqhpsuw.cpp:1400:26: warning: variable ‘tmp33’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:30:00.7258926Z 1400 | auto tmp33 = at::vec::convert(tmp32); 2024-08-20T22:30:00.7259448Z | ^~~~~ 2024-08-20T22:30:00.7261890Z /tmp/tmp3ydr6m14/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cles4fqzh5bjmrfn66gfzlwx2uzzdoyzhal7u7ivuvnc5cqhpsuw.cpp: In function ‘void cpp_fused_add_native_group_norm_relu_13(bfloat16*, const bfloat16*, const bfloat16*, const float*, const float*, const float*, const float*, const float*, const float*, float*, float*, long int)’: 2024-08-20T22:30:00.7265279Z /tmp/tmp3ydr6m14/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cles4fqzh5bjmrfn66gfzlwx2uzzdoyzhal7u7ivuvnc5cqhpsuw.cpp:1882:26: warning: variable ‘tmp33’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:30:00.7266886Z 1882 | auto tmp33 = at::vec::convert(tmp32); 2024-08-20T22:30:00.7267619Z | ^~~~~ 2024-08-20T22:30:00.7270013Z /tmp/tmp3ydr6m14/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cles4fqzh5bjmrfn66gfzlwx2uzzdoyzhal7u7ivuvnc5cqhpsuw.cpp: In function ‘void cpp_fused_add_native_group_norm_relu_18(bfloat16*, const bfloat16*, const bfloat16*, const float*, const float*, const float*, const float*, const float*, const float*, float*, float*, long int)’: 2024-08-20T22:30:00.7273421Z /tmp/tmp3ydr6m14/c7vlo2hz3rnc3qhjjqsuc75ymy6xy3dvgncmypsfke2q43fvsxvb/cles4fqzh5bjmrfn66gfzlwx2uzzdoyzhal7u7ivuvnc5cqhpsuw.cpp:2263:22: warning: variable ‘tmp35’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:30:00.7275015Z 2263 | auto tmp35 = at::vec::convert(tmp34); 2024-08-20T22:30:00.7275652Z | ^~~~~ 2024-08-20T22:30:01.5196393Z pass 2024-08-20T22:30:01.5198762Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:01.5200422Z TIMING: backend_compile:9e-05 entire_frame_compile:1.76876 total_wall_time:1.76876 2024-08-20T22:30:01.5202700Z STATS: call_* op count: 69 | FakeTensor.__torch_dispatch__:1505 | FakeTensorMode.__torch_dispatch__:16690 | attempt fast:1021 | fast is_contiguous:729 | ProxyTorchDispatchMode.__torch_dispatch__:1581 | slow no contiguity match:196 | fast channels_last:96 2024-08-20T22:30:01.5204892Z Dynamo produced 1 graphs covering 69 ops with 0 graph breaks (0 unique) 2024-08-20T22:30:05.2106739Z 2024-08-20T22:30:05.3588073Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:05.3588557Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:05.3589004Z cpu eval functorch_maml_omniglot 2024-08-20T22:30:05.3641516Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:05.3690155Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:05.3729196Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:26.7226417Z /tmp/tmp3ydr6m14/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/cdjqdu3ot74fak7xtxigxwfyhzkyut6aq5s3khrj53pl25n5mghh.cpp: In function ‘void cpp_fused__native_batch_norm_legit_max_pool2d_with_indices_relu_1(const bfloat16*, const float*, const float*, float*, float*, bfloat16*, bfloat16*)’: 2024-08-20T22:30:26.7231121Z /tmp/tmp3ydr6m14/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/cdjqdu3ot74fak7xtxigxwfyhzkyut6aq5s3khrj53pl25n5mghh.cpp:494:36: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:30:26.7232742Z 494 | Welford tmp_acc0 = Welford(); 2024-08-20T22:30:26.7233331Z | ^~~~~~~~ 2024-08-20T22:30:26.7235551Z /tmp/tmp3ydr6m14/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/cdjqdu3ot74fak7xtxigxwfyhzkyut6aq5s3khrj53pl25n5mghh.cpp: In function ‘void cpp_fused__native_batch_norm_legit_max_pool2d_with_indices_relu_2(const bfloat16*, const float*, const float*, float*, float*, bfloat16*, bfloat16*)’: 2024-08-20T22:30:26.7238791Z /tmp/tmp3ydr6m14/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/cdjqdu3ot74fak7xtxigxwfyhzkyut6aq5s3khrj53pl25n5mghh.cpp:585:32: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:30:26.7240376Z 585 | Welford tmp_acc0 = Welford(); 2024-08-20T22:30:26.7240900Z | ^~~~~~~~ 2024-08-20T22:30:26.7243142Z /tmp/tmp3ydr6m14/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/cdjqdu3ot74fak7xtxigxwfyhzkyut6aq5s3khrj53pl25n5mghh.cpp: In function ‘void cpp_fused__native_batch_norm_legit_max_pool2d_with_indices_relu_3(const bfloat16*, const float*, const float*, float*, float*, bfloat16*, bfloat16*, bfloat16*)’: 2024-08-20T22:30:26.7246754Z /tmp/tmp3ydr6m14/c5ibghnh7eby2o3hpvfo6vtnfzg3fbkpe3mfac4tk2mvchtuwize/cdjqdu3ot74fak7xtxigxwfyhzkyut6aq5s3khrj53pl25n5mghh.cpp:677:32: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:30:26.7248353Z 677 | Welford tmp_acc0 = Welford(); 2024-08-20T22:30:26.7248877Z | ^~~~~~~~ 2024-08-20T22:30:26.9861016Z pass 2024-08-20T22:30:26.9862285Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:26.9863873Z TIMING: backend_compile:8e-05 entire_frame_compile:0.18756 total_wall_time:0.18756 2024-08-20T22:30:26.9865473Z STATS: call_* op count: 14 | FakeTensor.__torch_dispatch__:248 | FakeTensorMode.__torch_dispatch__:2140 | ProxyTorchDispatchMode.__torch_dispatch__:400 2024-08-20T22:30:26.9867286Z Dynamo produced 1 graphs covering 14 ops with 0 graph breaks (0 unique) 2024-08-20T22:30:30.2368037Z 2024-08-20T22:30:32.0133735Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:32.0134238Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:30:32.0134679Z cpu eval hf_Albert 2024-08-20T22:30:33.3236806Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:34.8080277Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:30:36.2831870Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:31:56.7382786Z pass 2024-08-20T22:31:56.7385367Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:31:56.7386384Z TIMING: backend_compile:0.00017 entire_frame_compile:6.36892 total_wall_time:6.36892 2024-08-20T22:31:56.7387861Z STATS: call_* op count: 438 | FakeTensorMode.__torch_dispatch__:41080 | attempt fast:3152 | fast is_contiguous:3152 | FakeTensor.__torch_dispatch__:3506 | ProxyTorchDispatchMode.__torch_dispatch__:6974 2024-08-20T22:31:56.7389277Z Dynamo produced 1 graphs covering 438 ops with 0 graph breaks (0 unique) 2024-08-20T22:32:01.2340268Z 2024-08-20T22:32:05.8508736Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:32:05.8509230Z loading model: 0it [00:04, ?it/s] 2024-08-20T22:32:05.8509664Z cpu eval hf_Bart 2024-08-20T22:32:07.3669367Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:32:09.5092044Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:32:11.7376376Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:33:53.1415401Z pass 2024-08-20T22:33:53.1417996Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:33:53.1419155Z TIMING: backend_compile:0.00019 entire_frame_compile:6.42848 total_wall_time:6.42848 2024-08-20T22:33:53.1420789Z STATS: call_* op count: 470 | FakeTensorMode.__torch_dispatch__:47443 | FakeTensor.__torch_dispatch__:4034 | attempt fast:3289 | fast is_contiguous:3288 | slow no contiguity match:1 | ProxyTorchDispatchMode.__torch_dispatch__:8066 2024-08-20T22:33:53.1422364Z Dynamo produced 1 graphs covering 470 ops with 0 graph breaks (0 unique) 2024-08-20T22:33:57.7706341Z 2024-08-20T22:34:00.9537412Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:34:00.9537929Z loading model: 0it [00:03, ?it/s] 2024-08-20T22:34:00.9538873Z cpu eval hf_Bert 2024-08-20T22:34:03.0827866Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:34:04.8547760Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:34:06.6464555Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:35:32.5777741Z pass 2024-08-20T22:35:32.5779025Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:35:32.5782393Z TIMING: backend_compile:0.00015 entire_frame_compile:7.60063 total_wall_time:7.60063 2024-08-20T22:35:32.5783913Z STATS: call_* op count: 368 | FakeTensorMode.__torch_dispatch__:40960 | attempt fast:2907 | fast is_contiguous:2907 | FakeTensor.__torch_dispatch__:3252 | ProxyTorchDispatchMode.__torch_dispatch__:7536 2024-08-20T22:35:32.5785324Z Dynamo produced 1 graphs covering 368 ops with 0 graph breaks (0 unique) 2024-08-20T22:35:37.1422280Z 2024-08-20T22:35:44.0865765Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:35:44.0866243Z loading model: 0it [00:06, ?it/s] 2024-08-20T22:35:44.0866682Z cpu eval hf_Bert_large 2024-08-20T22:35:49.0365385Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:35:53.5441103Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:35:58.0461855Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:38:47.1940390Z pass 2024-08-20T22:38:47.1942835Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:38:47.1944810Z TIMING: backend_compile:0.00024 entire_frame_compile:15.02889 total_wall_time:15.02889 2024-08-20T22:38:47.1947458Z STATS: call_* op count: 716 | FakeTensorMode.__torch_dispatch__:79888 | attempt fast:5643 | fast is_contiguous:5643 | FakeTensor.__torch_dispatch__:6324 | ProxyTorchDispatchMode.__torch_dispatch__:14712 2024-08-20T22:38:47.1950089Z Dynamo produced 1 graphs covering 716 ops with 0 graph breaks (0 unique) 2024-08-20T22:38:53.0266182Z 2024-08-20T22:38:55.2167312Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:38:55.2170722Z loading model: 0it [00:02, ?it/s] 2024-08-20T22:38:55.2174116Z cpu eval hf_DistilBert 2024-08-20T22:38:56.2136190Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:38:57.2207089Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:38:58.2306148Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:39:58.3942365Z pass 2024-08-20T22:39:58.3943148Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:39:58.3944323Z TIMING: backend_compile:0.00011 entire_frame_compile:3.36731 total_wall_time:3.36731 2024-08-20T22:39:58.3945943Z STATS: call_* op count: 205 | FakeTensor.__torch_dispatch__:2009 | FakeTensorMode.__torch_dispatch__:23889 | attempt fast:1565 | fast is_contiguous:1517 | slow no contiguity match:48 | ProxyTorchDispatchMode.__torch_dispatch__:3879 2024-08-20T22:39:58.3947533Z Dynamo produced 1 graphs covering 205 ops with 0 graph breaks (0 unique) 2024-08-20T22:40:02.3411185Z 2024-08-20T22:40:05.3845730Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:40:05.3846265Z loading model: 0it [00:03, ?it/s] 2024-08-20T22:40:05.3855742Z cpu eval hf_GPT2 2024-08-20T22:40:06.0660006Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:40:06.6591160Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:40:07.2299802Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:41:32.3280257Z pass 2024-08-20T22:41:32.3282649Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:41:32.3283963Z TIMING: backend_compile:0.00023 entire_frame_compile:6.57146 total_wall_time:6.57146 2024-08-20T22:41:32.3285470Z STATS: call_* op count: 623 | FakeTensorMode.__torch_dispatch__:37950 | FakeTensor.__torch_dispatch__:2531 | attempt fast:2710 | fast is_contiguous:2710 | ProxyTorchDispatchMode.__torch_dispatch__:6581 2024-08-20T22:41:32.3287164Z Dynamo produced 1 graphs covering 623 ops with 0 graph breaks (0 unique) 2024-08-20T22:41:36.9383908Z 2024-08-20T22:42:03.9041829Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:42:03.9044384Z loading model: 0it [00:26, ?it/s] 2024-08-20T22:42:03.9044878Z cpu eval hf_GPT2_large 2024-08-20T22:42:03.9045397Z pass_due_to_skip 2024-08-20T22:42:03.9048429Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:42:03.9049453Z TIMING: 2024-08-20T22:42:03.9050029Z STATS: call_* op count: 0 2024-08-20T22:42:03.9050613Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:42:07.2619120Z 2024-08-20T22:42:39.1892526Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:42:39.1900038Z loading model: 0it [00:31, ?it/s] 2024-08-20T22:42:39.1901822Z cpu eval hf_T5_base 2024-08-20T22:43:21.5007672Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:44:11.1285473Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:45:01.9609724Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:49:05.1536215Z pass 2024-08-20T22:49:05.1536997Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:49:05.1540925Z TIMING: backend_compile:0.00041 entire_frame_compile:15.23736 total_wall_time:15.23736 2024-08-20T22:49:05.1542856Z STATS: call_* op count: 1516 | FakeTensorMode.__torch_dispatch__:93330 | FakeTensor.__torch_dispatch__:8649 | attempt fast:3332 | fast is_contiguous:3164 | slow both tensors nontrivially broadcast:24 | slow no contiguity match:144 | ProxyTorchDispatchMode.__torch_dispatch__:17886 2024-08-20T22:49:05.1544935Z Dynamo produced 1 graphs covering 1516 ops with 0 graph breaks (0 unique) 2024-08-20T22:49:11.8363172Z 2024-08-20T22:49:29.6767421Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:49:29.6767933Z loading model: 0it [00:17, ?it/s] 2024-08-20T22:49:29.6768608Z cpu eval hf_T5_large 2024-08-20T22:49:29.6770846Z pass_due_to_skip 2024-08-20T22:49:29.6771561Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:49:29.6772285Z TIMING: 2024-08-20T22:49:29.6772537Z STATS: call_* op count: 0 2024-08-20T22:49:29.6773076Z Dynamo produced 0 graphs covering 0 ops with 0 graph breaks (0 unique) 2024-08-20T22:49:33.1200676Z 2024-08-20T22:49:44.0965653Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:49:44.0966982Z loading model: 0it [00:10, ?it/s] 2024-08-20T22:49:44.0967839Z cpu eval hf_distil_whisper 2024-08-20T22:49:53.3104815Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:50:06.1386112Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:50:19.1092815Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:52:34.0892737Z /tmp/tmp3ydr6m14/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/cw6kgozooi5qldmeieleho2on54yv4rmu3lsgadigw7zkvpn6lsv.cpp: In function ‘void cpp_fused__to_copy_add_native_layer_norm_2(const bfloat16*, const float*, const float*, const float*, float*, float*, bfloat16*, long int)’: 2024-08-20T22:52:34.0896059Z /tmp/tmp3ydr6m14/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/cw6kgozooi5qldmeieleho2on54yv4rmu3lsgadigw7zkvpn6lsv.cpp:535:40: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:52:34.0897674Z 535 | Welford tmp_acc0 = Welford(); 2024-08-20T22:52:34.0898272Z | ^~~~~~~~ 2024-08-20T22:52:34.0901359Z /tmp/tmp3ydr6m14/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/cw6kgozooi5qldmeieleho2on54yv4rmu3lsgadigw7zkvpn6lsv.cpp: In function ‘void cpp_fused__to_copy_add_native_layer_norm_4(const bfloat16*, const float*, const bfloat16*, const float*, const float*, float*, float*, float*, bfloat16*, long int)’: 2024-08-20T22:52:34.0904666Z /tmp/tmp3ydr6m14/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/cw6kgozooi5qldmeieleho2on54yv4rmu3lsgadigw7zkvpn6lsv.cpp:782:40: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:52:34.0906244Z 782 | Welford tmp_acc0 = Welford(); 2024-08-20T22:52:34.0906787Z | ^~~~~~~~ 2024-08-20T22:52:34.0909153Z /tmp/tmp3ydr6m14/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/cw6kgozooi5qldmeieleho2on54yv4rmu3lsgadigw7zkvpn6lsv.cpp: In function ‘void cpp_fused__to_copy_add_native_layer_norm_5(const bfloat16*, const float*, const bfloat16*, const bfloat16*, const float*, const float*, float*, float*, float*, bfloat16*, long int)’: 2024-08-20T22:52:34.0915372Z /tmp/tmp3ydr6m14/ccqajjok6zmfbyff2r72d4gfeldumigbkxhx66iztlqxbumpzjhi/cw6kgozooi5qldmeieleho2on54yv4rmu3lsgadigw7zkvpn6lsv.cpp:984:40: warning: variable ‘tmp_acc0’ set but not used [-Wunused-but-set-variable] 2024-08-20T22:52:34.0916987Z 984 | Welford tmp_acc0 = Welford(); 2024-08-20T22:52:34.0917517Z | ^~~~~~~~ 2024-08-20T22:53:05.0381305Z pass 2024-08-20T22:53:05.0383335Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:53:05.0384362Z TIMING: backend_compile:0.00024 entire_frame_compile:9.37734 total_wall_time:9.37734 2024-08-20T22:53:05.0386015Z STATS: call_* op count: 660 | FakeTensor.__torch_dispatch__:6107 | FakeTensorMode.__torch_dispatch__:78114 | attempt fast:5534 | fast is_contiguous:5093 | slow no contiguity match:441 | ProxyTorchDispatchMode.__torch_dispatch__:11806 2024-08-20T22:53:05.0387616Z Dynamo produced 1 graphs covering 660 ops with 0 graph breaks (0 unique) 2024-08-20T22:53:10.7302414Z 2024-08-20T22:53:10.8430217Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:53:10.8430724Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:53:10.8431174Z cpu eval lennard_jones 2024-08-20T22:53:10.8461945Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:53:10.8487311Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:53:10.8500949Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:53:31.3479730Z pass 2024-08-20T22:53:31.3483314Z WARNING:common:Trying to call the empty_gpu_cache for device: cpu, which is not in list [cuda, xpu] 2024-08-20T22:53:31.3484639Z TIMING: backend_compile:0.0001 entire_frame_compile:0.2545 total_wall_time:0.2545 2024-08-20T22:53:31.3486514Z STATS: call_* op count: 9 | FakeTensor.__torch_dispatch__:91 | FakeTensorMode.__torch_dispatch__:1829 | attempt fast:70 | fast is_contiguous:70 | ProxyTorchDispatchMode.__torch_dispatch__:299 2024-08-20T22:53:31.3488298Z Dynamo produced 1 graphs covering 9 ops with 0 graph breaks (0 unique) 2024-08-20T22:53:32.2741773Z accuracy pass_rate=60.00% 2024-08-20T22:53:32.2746337Z calls_captured gmean=0.00x mean=183.714x 2024-08-20T22:53:32.2750412Z unique_graphs gmean=0.00x mean=0.657x 2024-08-20T22:53:32.2755276Z graph_breaks gmean=0.00x mean=0.000x 2024-08-20T22:53:32.2758436Z unique_graph_breaks gmean=0.00x mean=0.000x 2024-08-20T22:53:32.2762088Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T22:53:32.2765717Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T22:53:32.2769263Z cudagraph_skips gmean=0.00x mean=0.000x 2024-08-20T22:53:33.7711924Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamic_cpu_aot_inductor_torchbench_amp_freezing_inference.csv 2024-08-20T22:53:34.1087460Z BERT_pytorch XFAIL 2024-08-20T22:53:34.1088080Z Background_Matting XFAIL 2024-08-20T22:53:34.1090396Z LearningToPaint PASS 2024-08-20T22:53:34.1095192Z Super_SloMo PASS 2024-08-20T22:53:34.1100205Z alexnet PASS 2024-08-20T22:53:34.1105140Z basic_gnn_edgecnn PASS 2024-08-20T22:53:34.1109951Z basic_gnn_gcn PASS 2024-08-20T22:53:34.1114812Z basic_gnn_gin PASS 2024-08-20T22:53:34.1119606Z basic_gnn_sage PASS 2024-08-20T22:53:34.1124500Z dcgan PASS 2024-08-20T22:53:34.1129259Z demucs PASS 2024-08-20T22:53:34.1134189Z densenet121 PASS 2024-08-20T22:53:34.1139024Z detectron2_fasterrcnn_r_101_c4 XFAIL 2024-08-20T22:53:34.1144388Z detectron2_fasterrcnn_r_101_dc5 XFAIL 2024-08-20T22:53:34.1149284Z detectron2_fasterrcnn_r_101_fpn XFAIL 2024-08-20T22:53:34.1154176Z detectron2_fasterrcnn_r_50_c4 XFAIL 2024-08-20T22:53:34.1158978Z detectron2_fasterrcnn_r_50_dc5 XFAIL 2024-08-20T22:53:34.1163808Z detectron2_fasterrcnn_r_50_fpn XFAIL 2024-08-20T22:53:34.1168634Z dlrm XFAIL 2024-08-20T22:53:34.1173502Z doctr_det_predictor XFAIL 2024-08-20T22:53:34.1178318Z doctr_reco_predictor XFAIL 2024-08-20T22:53:34.1183342Z drq XFAIL 2024-08-20T22:53:34.1188145Z functorch_dp_cifar10 PASS 2024-08-20T22:53:34.1192996Z functorch_maml_omniglot PASS 2024-08-20T22:53:34.1197775Z hf_Albert PASS 2024-08-20T22:53:34.1202559Z hf_Bart PASS 2024-08-20T22:53:34.1207364Z hf_Bert PASS 2024-08-20T22:53:34.1212199Z hf_Bert_large PASS 2024-08-20T22:53:34.1217038Z hf_DistilBert PASS 2024-08-20T22:53:34.1222056Z hf_GPT2 PASS 2024-08-20T22:53:34.1226875Z hf_GPT2_large XFAIL 2024-08-20T22:53:34.1231653Z hf_T5_base PASS 2024-08-20T22:53:34.1236479Z hf_T5_large XFAIL 2024-08-20T22:53:34.1241299Z hf_distil_whisper PASS 2024-08-20T22:53:34.1247149Z lennard_jones PASS 2024-08-20T22:53:34.1634565Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_torchbench.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamic_cpu_aot_inductor_torchbench_amp_freezing_inference.csv 2024-08-20T22:53:34.4902429Z BERT_pytorch PASS 2024-08-20T22:53:34.4903888Z Background_Matting PASS 2024-08-20T22:53:34.4908460Z LearningToPaint PASS 2024-08-20T22:53:34.4913281Z Super_SloMo PASS 2024-08-20T22:53:34.4918008Z alexnet PASS 2024-08-20T22:53:34.4922807Z basic_gnn_edgecnn PASS 2024-08-20T22:53:34.4927574Z basic_gnn_gcn PASS 2024-08-20T22:53:34.4932343Z basic_gnn_gin PASS 2024-08-20T22:53:34.4937111Z basic_gnn_sage PASS 2024-08-20T22:53:34.4942134Z dcgan PASS 2024-08-20T22:53:34.4947455Z demucs PASS 2024-08-20T22:53:34.4952211Z densenet121 PASS 2024-08-20T22:53:34.4957008Z detectron2_fasterrcnn_r_101_c4 PASS 2024-08-20T22:53:34.4961756Z detectron2_fasterrcnn_r_101_dc5 PASS 2024-08-20T22:53:34.4966543Z detectron2_fasterrcnn_r_101_fpn PASS 2024-08-20T22:53:34.4971347Z detectron2_fasterrcnn_r_50_c4 PASS 2024-08-20T22:53:34.4976115Z detectron2_fasterrcnn_r_50_dc5 PASS 2024-08-20T22:53:34.4981117Z detectron2_fasterrcnn_r_50_fpn PASS 2024-08-20T22:53:34.4985896Z dlrm PASS 2024-08-20T22:53:34.4990673Z doctr_det_predictor PASS 2024-08-20T22:53:34.4995716Z doctr_reco_predictor PASS 2024-08-20T22:53:34.5000194Z drq PASS 2024-08-20T22:53:34.5004977Z functorch_dp_cifar10 PASS 2024-08-20T22:53:34.5009764Z functorch_maml_omniglot PASS 2024-08-20T22:53:34.5014518Z hf_Albert PASS 2024-08-20T22:53:34.5019267Z hf_Bart PASS 2024-08-20T22:53:34.5024299Z hf_Bert PASS 2024-08-20T22:53:34.5029069Z hf_Bert_large PASS 2024-08-20T22:53:34.5033852Z hf_DistilBert PASS 2024-08-20T22:53:34.5038607Z hf_GPT2 PASS 2024-08-20T22:53:34.5043539Z hf_GPT2_large PASS 2024-08-20T22:53:34.5048513Z hf_T5_base PASS 2024-08-20T22:53:34.5053273Z hf_T5_large PASS 2024-08-20T22:53:34.5058053Z hf_distil_whisper PASS 2024-08-20T22:53:34.5063046Z lennard_jones PASS 2024-08-20T22:53:34.5448661Z + cleanup_workspace 2024-08-20T22:53:34.5449791Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T22:53:34.5450952Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T22:53:34.5451924Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T22:53:34.5452579Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T22:53:34.5453419Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T22:53:34.5454355Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T22:53:34.5455079Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T22:53:35.0099696Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T22:53:35.0100220Z cat test/**/*_toprint.log || true 2024-08-20T22:53:35.0109258Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:53:35.0109746Z env: 2024-08-20T22:53:35.0109987Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:35.0110758Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:35.0111517Z ##[endgroup] 2024-08-20T22:53:35.0185020Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T22:53:35.0212304Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T22:53:35.0212758Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T22:53:35.0218199Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:53:35.0218664Z env: 2024-08-20T22:53:35.0218908Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:35.0219478Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:35.0220396Z MONITOR_SCRIPT_PID: 43664 2024-08-20T22:53:35.0220788Z ##[endgroup] 2024-08-20T22:53:35.0429762Z Prepare all required actions 2024-08-20T22:53:35.0430248Z Getting action download info 2024-08-20T22:53:35.2509210Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T22:53:35.4398164Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T22:53:35.4398625Z with: 2024-08-20T22:53:35.4399266Z file-suffix: test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933 2024-08-20T22:53:35.4400022Z s3-bucket: gha-artifacts 2024-08-20T22:53:35.4400349Z env: 2024-08-20T22:53:35.4400604Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:35.4401186Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:35.4401830Z ##[endgroup] 2024-08-20T22:53:35.4425682Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T22:53:35.4426313Z # Remove any previous test jsons if they exist 2024-08-20T22:53:35.4426861Z rm -f test-jsons-*.zip 2024-08-20T22:53:35.4427367Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T22:53:35.4433129Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:53:35.4433616Z env: 2024-08-20T22:53:35.4433876Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:35.4434461Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:35.4435579Z FILE_SUFFIX: test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933 2024-08-20T22:53:35.4436285Z ##[endgroup] 2024-08-20T22:53:35.4731410Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T22:53:35.4779628Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T22:53:35.4782596Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T22:53:35.4802118Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T22:53:35.4810550Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T22:53:35.4837890Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T22:53:35.4838525Z # Remove any previous test reports if they exist 2024-08-20T22:53:35.4839038Z rm -f test-reports-*.zip 2024-08-20T22:53:35.4839611Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T22:53:35.4845623Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:53:35.4846108Z env: 2024-08-20T22:53:35.4846370Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:35.4846972Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:35.4848012Z FILE_SUFFIX: test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933 2024-08-20T22:53:35.4848767Z ##[endgroup] 2024-08-20T22:53:35.5028568Z adding: test/test-reports/inference_torchbench.csv (deflated 72%) 2024-08-20T22:53:35.5052533Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T22:53:35.5053159Z # Remove any previous usage logs if they exist 2024-08-20T22:53:35.5053651Z rm -f logs-*.zip 2024-08-20T22:53:35.5054325Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T22:53:35.5055130Z # so check to see if the file exists first 2024-08-20T22:53:35.5055610Z if [ -f 'usage_log.txt' ]; then 2024-08-20T22:53:35.5056232Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T22:53:35.5056795Z fi 2024-08-20T22:53:35.5057115Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T22:53:35.5057799Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T22:53:35.5058270Z fi 2024-08-20T22:53:35.5063677Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:53:35.5064174Z env: 2024-08-20T22:53:35.5064439Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:35.5065186Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:35.5066329Z FILE_SUFFIX: test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933 2024-08-20T22:53:35.5067038Z ##[endgroup] 2024-08-20T22:53:35.5137418Z adding: usage_log.txt (deflated 92%) 2024-08-20T22:53:35.5680275Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T22:53:35.5680950Z # Remove any previous debugging artifacts if they exist 2024-08-20T22:53:35.5681446Z rm -f debug-*.zip 2024-08-20T22:53:35.5681787Z if [ -d 'test/debug' ]; then 2024-08-20T22:53:35.5682247Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T22:53:35.5682679Z fi 2024-08-20T22:53:35.5687299Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:53:35.5687766Z env: 2024-08-20T22:53:35.5688008Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:35.5688581Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:35.5689535Z FILE_SUFFIX: test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933 2024-08-20T22:53:35.5690245Z ##[endgroup] 2024-08-20T22:53:35.5770598Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:53:35.5771018Z with: 2024-08-20T22:53:35.5771377Z s3-bucket: gha-artifacts 2024-08-20T22:53:35.5771788Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:53:35.5772240Z retention-days: 14 2024-08-20T22:53:35.5772533Z if-no-files-found: warn 2024-08-20T22:53:35.5772866Z path: test-jsons-*.zip 2024-08-20T22:53:35.5773178Z name: artifact 2024-08-20T22:53:35.5773455Z region: us-east-1 2024-08-20T22:53:35.5773718Z env: 2024-08-20T22:53:35.5773961Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:35.5774684Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:35.5796469Z ##[endgroup] 2024-08-20T22:53:35.9428899Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:53:35.9429505Z With the provided path, there will be 1 file uploaded 2024-08-20T22:53:35.9430129Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:53:36.0759630Z Starting upload of test-jsons-test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933.zip 2024-08-20T22:53:36.2389092Z Finished upload of test-jsons-test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933.zip 2024-08-20T22:53:36.2613722Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:53:36.2614177Z with: 2024-08-20T22:53:36.2614434Z s3-bucket: gha-artifacts 2024-08-20T22:53:36.2614872Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:53:36.2615348Z retention-days: 14 2024-08-20T22:53:36.2615681Z if-no-files-found: error 2024-08-20T22:53:36.2616044Z path: test-reports-*.zip 2024-08-20T22:53:36.2616390Z name: artifact 2024-08-20T22:53:36.2616689Z region: us-east-1 2024-08-20T22:53:36.2616977Z env: 2024-08-20T22:53:36.2617242Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:36.2617858Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:36.2618530Z ##[endgroup] 2024-08-20T22:53:36.5926509Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:53:36.5927134Z With the provided path, there will be 1 file uploaded 2024-08-20T22:53:36.5927744Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:53:36.5966006Z Starting upload of test-reports-test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933.zip 2024-08-20T22:53:36.7323605Z Finished upload of test-reports-test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933.zip 2024-08-20T22:53:36.7529826Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:53:36.7530226Z with: 2024-08-20T22:53:36.7530484Z s3-bucket: gha-artifacts 2024-08-20T22:53:36.7531045Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:53:36.7531493Z retention-days: 14 2024-08-20T22:53:36.7531784Z if-no-files-found: ignore 2024-08-20T22:53:36.7532114Z path: logs-*.zip 2024-08-20T22:53:36.7532396Z name: artifact 2024-08-20T22:53:36.7532652Z region: us-east-1 2024-08-20T22:53:36.7532922Z env: 2024-08-20T22:53:36.7533164Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:36.7533727Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:36.7534326Z ##[endgroup] 2024-08-20T22:53:37.0813924Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:53:37.0814714Z With the provided path, there will be 1 file uploaded 2024-08-20T22:53:37.0815336Z Uploading to s3 prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:53:37.0852574Z Starting upload of logs-test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933.zip 2024-08-20T22:53:37.2063774Z Finished upload of logs-test-dynamic_cpu_aot_inductor_torchbench_amp_freezing-1-2-amz2023.linux.12xlarge_29025551933.zip 2024-08-20T22:53:37.2267930Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:53:37.2268334Z with: 2024-08-20T22:53:37.2268586Z s3-bucket: gha-artifacts 2024-08-20T22:53:37.2268988Z s3-prefix: pytorch/pytorch/10479310725/1/artifact 2024-08-20T22:53:37.2269425Z retention-days: 14 2024-08-20T22:53:37.2269713Z if-no-files-found: ignore 2024-08-20T22:53:37.2270036Z path: debug-*.zip 2024-08-20T22:53:37.2270315Z name: artifact 2024-08-20T22:53:37.2270568Z region: us-east-1 2024-08-20T22:53:37.2270832Z env: 2024-08-20T22:53:37.2271074Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:37.2271640Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:37.2272238Z ##[endgroup] 2024-08-20T22:53:37.5516432Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T22:53:37.5701365Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T22:53:37.5701832Z # shellcheck disable=SC2156 2024-08-20T22:53:37.5702649Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T22:53:37.5708654Z shell: /usr/bin/bash -e {0} 2024-08-20T22:53:37.5708986Z env: 2024-08-20T22:53:37.5709247Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:37.5709852Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:37.5710496Z ##[endgroup] 2024-08-20T22:53:37.7928357Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T22:53:37.7929059Z with: 2024-08-20T22:53:37.7929287Z env: 2024-08-20T22:53:37.7929534Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:37.7930108Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:37.7930759Z ##[endgroup] 2024-08-20T22:53:37.7967541Z ##[group]Run set -eou pipefail 2024-08-20T22:53:37.7967930Z set -eou pipefail 2024-08-20T22:53:37.7968263Z  2024-08-20T22:53:37.7968772Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T22:53:37.7969407Z for _ in $(seq 1440); do 2024-08-20T22:53:37.7969865Z  # Break if no ssh session exists anymore 2024-08-20T22:53:37.7970353Z  if [ "$(who)" = "" ]; then 2024-08-20T22:53:37.7970751Z  break 2024-08-20T22:53:37.7971029Z  fi 2024-08-20T22:53:37.7971353Z  echo "." 2024-08-20T22:53:37.7971650Z  sleep 5 2024-08-20T22:53:37.7971946Z done 2024-08-20T22:53:37.7977435Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:53:37.7977922Z env: 2024-08-20T22:53:37.7978164Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:37.7978857Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:37.7979687Z ##[endgroup] 2024-08-20T22:53:37.8002732Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T22:53:37.8076878Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:53:37.8077630Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:53:37.8078210Z # shellcheck disable=SC2046 2024-08-20T22:53:37.8078647Z docker stop $(docker ps -q) || true 2024-08-20T22:53:37.8079188Z # Prune all of the docker images 2024-08-20T22:53:37.8079598Z docker system prune -af 2024-08-20T22:53:37.8084448Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:53:37.8084908Z env: 2024-08-20T22:53:37.8085139Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:53:37.8085698Z DOCKER_CONTAINER_ID: 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:37.8086305Z ##[endgroup] 2024-08-20T22:53:48.6997727Z 8f9765e7049f 2024-08-20T22:53:50.2632395Z Deleted Containers: 2024-08-20T22:53:50.2632997Z 8f9765e7049f25b4ab8136ef7ca66270dd7e08c9438366ff06ef16212867b344 2024-08-20T22:53:50.2633451Z 2024-08-20T22:53:54.0729765Z Deleted Images: 2024-08-20T22:53:54.0731328Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:53:54.0733858Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11-inductor-benchmarks@sha256:ed18f711327455e269271a17448384275684348a472ba289f00904bd14501fa9 2024-08-20T22:53:54.0735460Z deleted: sha256:822356886a68936cce34917f5ffc60d864918fab7594bb08c26bf1816dd6110d 2024-08-20T22:53:54.0736485Z deleted: sha256:ae6b018f5c327102b48941d39cd8d6fa6151893715290829a76e4e73add3a418 2024-08-20T22:53:54.0737379Z deleted: sha256:79fa7dddbf1ed3af5e8f10972d07ccea774929efe8578a1cc6604a00c93c1e02 2024-08-20T22:53:54.0738488Z deleted: sha256:06b439d80ae875bb9b96d0a23c4619aec5b5fb2f35b5048022ebe6061e19f7e1 2024-08-20T22:53:54.0739480Z deleted: sha256:52d3d63b327ae120e60f76fe27965e31638eb8d556b5e5f56f4773bd940373cf 2024-08-20T22:53:54.0740334Z deleted: sha256:5df173ad6b0f1ba16287832b6e5237513d152e87e627e478fc36ed16159697c0 2024-08-20T22:53:54.0741303Z deleted: sha256:b6c13bf75251ea08468f4040dbba32848f72cd00eb8c3003ecc1e879f73738ae 2024-08-20T22:53:54.0742190Z deleted: sha256:f2f37b6c898fc6a8c74683cc50a156f56abe71f20310c72338bb0ec23c71e8d6 2024-08-20T22:53:54.0743081Z deleted: sha256:55832a6aecdb11a84ddbbd38efcac3b62dd61a2b7832a1080535fc5730e1e336 2024-08-20T22:53:54.0756157Z deleted: sha256:bd86c04ac5f8ba1940bc991dfbae9e17a80293b5cf5fe568c694f46d59863011 2024-08-20T22:53:54.0757095Z deleted: sha256:88a76e0add806e67ed7bfd8ad7f862a8683d087dd3557cfc2b787da5f4c1ae0f 2024-08-20T22:53:54.0757989Z deleted: sha256:37a2749f2e5a556bb0369b3db12c47e4b82b41064f430896babceb49d97d2f9f 2024-08-20T22:53:54.0758853Z deleted: sha256:ac6214a417e93e76048bee154136a3117823ef38824b38f2659c74c7d09b57f8 2024-08-20T22:53:54.0759732Z deleted: sha256:bd60508bf2fcb98670e5ba90896321868f6fb9c5a2c859203e47512542f45b99 2024-08-20T22:53:54.0760704Z deleted: sha256:3fe3c9441ff3190f5cf87c066d3b5ca03aa3bb68071ef86e127c271ebb1ea02e 2024-08-20T22:53:54.0761718Z deleted: sha256:f9b8978a3de27376176f0400dcb17127771f2a8123ff323474e055dbae6eaba5 2024-08-20T22:53:54.0762581Z deleted: sha256:825e6cc758ce8478a096de8d367123d4d0a7f51fbe595867d2f950f714ca908e 2024-08-20T22:53:54.0763458Z deleted: sha256:dde33dfb74ab43131553ba775cd8561874596ba2e3d1b810c7796beba3d4fd38 2024-08-20T22:53:54.0764325Z deleted: sha256:809173faaa7c1289371a0c5a0927a6050de23417cddba0746dea79b10b68dd71 2024-08-20T22:53:54.0765229Z deleted: sha256:f63fee6ae14582933c75117b8d3115041d0177792e510d8dab0baca1b2202771 2024-08-20T22:53:54.0766142Z deleted: sha256:9432a4a5f47ddff6148fd5f8ade43d3a5566c0da2fbcd904daa917dc21c4c73d 2024-08-20T22:53:54.0767050Z deleted: sha256:b9ff51a3d1dc73e8a3ab751fa1a512026a101eaedbea74cfe01a2876b5111502 2024-08-20T22:53:54.0767937Z deleted: sha256:70e4013b779cc78a919225cad0a55d38948b0620ea119eb80dbb377ce8687f7e 2024-08-20T22:53:54.0768790Z deleted: sha256:edcf423609137384d678c9f677cae55933016f3ee76cc2ee47d93aad1346c427 2024-08-20T22:53:54.0769669Z deleted: sha256:bb2f6a3359e809db17e8b77499bf0d94200af9defc470b3adb3e1c28a6f1c6c1 2024-08-20T22:53:54.0770539Z deleted: sha256:eb646c0b0638f760f7c1046a113d3646e57513d19390881f4f6775ec702fa265 2024-08-20T22:53:54.0771398Z deleted: sha256:71c5137ede0421f7f00b1e93ee7bd6287c8ff6445615d0e1d1edddff2240789c 2024-08-20T22:53:54.0772250Z deleted: sha256:d2a20794246965e241eaaa98883ecd02a39901618f06536532bb4d9ed2b80f63 2024-08-20T22:53:54.0773104Z deleted: sha256:a8fc5137968c5840b35ff322f4796356303c4d4f0ed5c86c70218e20045936ac 2024-08-20T22:53:54.0773968Z deleted: sha256:1bd367d8cc65cb8808ddf7665064a0523467c7947ead2e1aed5112e9465e2fa8 2024-08-20T22:53:54.0774840Z deleted: sha256:736b605a3af6c5f2f82b517f0c1c517701a78def165ad99f3fb034198818994d 2024-08-20T22:53:54.0775721Z deleted: sha256:f2766fd6c51e0f9c3fdeed91ff144d049ed99f1a2cdce5c3b36a9f694af46045 2024-08-20T22:53:54.0776593Z deleted: sha256:688dce500613b1366475010344298a9b4eca441474fb871cfc37287a6eefbbf4 2024-08-20T22:53:54.0777448Z deleted: sha256:08f9d176a447289b58f9c70cdc76546487fc2700b1e5a7b79c19eb4f8f7c4608 2024-08-20T22:53:54.0778328Z deleted: sha256:538a2fc18bbc7da30a73c02bc8bb61bd6800a9d501caeaec614a32c41d53438d 2024-08-20T22:53:54.0779312Z deleted: sha256:75d7a20f4c66e80cf12fdf6d68787ee75c33478fd7c3e30c4696702ef9932993 2024-08-20T22:53:54.0780183Z deleted: sha256:5f36a35d97f9c931857392a1a616a5af4da00dfabe321b21f7637761d680b575 2024-08-20T22:53:54.0781070Z deleted: sha256:172cb1246a46e9dde7d50405f7fb31fd83a3a3e6ff40c811cfe100bd034807f6 2024-08-20T22:53:54.0781955Z deleted: sha256:d621ca4d3a5a15f4d74797b18bd0f4c44f63288c18e753fc4281cec69d82ea21 2024-08-20T22:53:54.0782842Z deleted: sha256:8e5eec944591edb185a54d6b495949dcae1da0be3fec8d30c3463e9fcde1a34e 2024-08-20T22:53:54.0783708Z deleted: sha256:656151b8c659bb4dab88293de022523e0e478e654f6922eaf53efb36ffb26e7e 2024-08-20T22:53:54.0784666Z deleted: sha256:9e2c8ec2c392f561f1016a988a64620f377203ed895815f024856cdef934be6b 2024-08-20T22:53:54.0785534Z deleted: sha256:c42f2e17dc37198a178852ecf9d947775b76af99ebdc2aa929af4b03b3896b3b 2024-08-20T22:53:54.0786399Z deleted: sha256:f52f63d8f16c18cde14e5738947c6e08323452e8599f723a6e041ed1ab33565a 2024-08-20T22:53:54.0787259Z deleted: sha256:343dc9f9b3e3c25dc528bbe1f8b4258289e512ff85f4b9c22ea2154c9c9d8143 2024-08-20T22:53:54.0788147Z deleted: sha256:5efa5da92a100a8c36c51a732af9534a8023041baf9d55ecf831c1386467d65a 2024-08-20T22:53:54.0789110Z deleted: sha256:da5cd90133fe64eb015b47ab4515d89305598acefc5c097b8e189da64be2d476 2024-08-20T22:53:54.0789987Z deleted: sha256:691e04884c858a1ea0a97381a7b1036bdfc6f9a48471608f0a5db9da9935555f 2024-08-20T22:53:54.0790844Z deleted: sha256:64100fb171394a71add5752ea3f6c432bcba0ac31aca67d8443c3722864804e1 2024-08-20T22:53:54.0791767Z deleted: sha256:a03197ec2ed41935ef46795a87ffd3d80b526fce1edf28550177f579cfb5df64 2024-08-20T22:53:54.0792582Z deleted: sha256:7bd9c2008289af25e772b1fc7b8e6470b513a5f76ce25743b9ef1bbe43ace5f7 2024-08-20T22:53:54.0793381Z deleted: sha256:0064821cdad60c711279a5964f88bf82c7582c296d21a0d1d0e97c60a47c1fa3 2024-08-20T22:53:54.0794161Z deleted: sha256:5825eb9623523140ad1e3ad683f3a83ab39054adf0b0fb4e472c2a7c3e7545ff 2024-08-20T22:53:54.0794977Z deleted: sha256:ff2d63ec14fa462e2edcd48d8ca8b678d79c9981bda17880818db15b8790e54b 2024-08-20T22:53:54.0795803Z deleted: sha256:79a5278d4bd098095d2f18fcfa476e8d54112dae4ac3d6c4c955e7fd9ba550f4 2024-08-20T22:53:54.0796602Z deleted: sha256:808ef6924cb221e2093f74413950d96ff62707fc13a194d83a0422315fccccdc 2024-08-20T22:53:54.0797393Z deleted: sha256:91aa45c12c4226ea9961e3acd079df4454bf2a55cb1bb9a2918f37bec89947bf 2024-08-20T22:53:54.0798202Z deleted: sha256:1f26a67abbf110b4883c6dd39e035f74248d28916c4a48be34296a7a96d1319b 2024-08-20T22:53:54.0799013Z deleted: sha256:1165de789bc53191b61c2b04dc04cf9d3d6d78f3d704776bd5a276d2caac8d09 2024-08-20T22:53:54.0799850Z deleted: sha256:aff63be2ff8d27d7172fb6dbecff0bea191a85c2431f8de65c0f81d40212587b 2024-08-20T22:53:54.0800680Z deleted: sha256:7c4290bd5a95669ca4d362a8672cb96b80693cc40af13f1ce061e86f835766d1 2024-08-20T22:53:54.0801482Z deleted: sha256:46ad9cfd3aa9b54d845ebc37c2e4c0d5fda90577227a7962049b71c825acdafe 2024-08-20T22:53:54.0802314Z deleted: sha256:aad12f955a8ca5766f6ca5d1467a6d2b81be713ff7224e8ae8eb92b68acf69a2 2024-08-20T22:53:54.0803138Z deleted: sha256:f5a053a7d9bda27126802be2ba52d187c26e934b7cc582850d2aaadd90631a1e 2024-08-20T22:53:54.0803949Z deleted: sha256:cf0f3c2eb852401868a3f959f6176f5a6b42f17f16e950f132f03293c764d733 2024-08-20T22:53:54.0804716Z deleted: sha256:7929c61352934651542ff23122529355349b2153871117dcfbaa4b45020a158d 2024-08-20T22:53:54.0805499Z deleted: sha256:119390db59d077c75caa11c49fee6ab586d061d27de1deeeec8608f330624f21 2024-08-20T22:53:54.0806306Z deleted: sha256:3b7424223daa8c5790ef526174f3780d9b1b54baa45d080127bfc72e60139a2b 2024-08-20T22:53:54.0807104Z deleted: sha256:f8986b3dfad5395410b72709f00789a62ee60d1b0bc1b27583f4c17af34b43fc 2024-08-20T22:53:54.0807891Z deleted: sha256:03db7655e84b428b1399087a9ed34180c120808cd00d94924a46eeaef73c37b0 2024-08-20T22:53:54.0808887Z deleted: sha256:b4a81bb8166fcf8ffbf063817b3202900c20586ca4bf369b21822764ab966164 2024-08-20T22:53:54.0809745Z deleted: sha256:afc241c47ba97b4a4f054cd0e6a5d90428b2675d00b160727187b7feabb81b14 2024-08-20T22:53:54.0810609Z deleted: sha256:e04730e6f5fc53e139356efef9c2c84b7bf6087c5a89876deaf1be58b9b1029b 2024-08-20T22:53:54.0811463Z deleted: sha256:7b22c8a2bc4bb0051ddc3ea51c917e7562663e4dafe4f9ad4a3870f4ec84bd14 2024-08-20T22:53:54.0812326Z deleted: sha256:c54b28287358751c9e19bd4f52b1d920ec5f4a254f8883b9d093528c08bc8b3f 2024-08-20T22:53:54.0813172Z deleted: sha256:17303ef9705fd22b570529dc8413ccba1a59fab7f5fef103bfe72e01c53d976e 2024-08-20T22:53:54.0814033Z deleted: sha256:1a4e7403586e712f2caf32f8bec553dafd88f78612fba5bd0c54e4efb89b9c41 2024-08-20T22:53:54.0814875Z deleted: sha256:d2f2462c603417aa43b82ea6f370ed4319cd12b3bda370d5a7ada0845daf3aaa 2024-08-20T22:53:54.0815805Z deleted: sha256:973bcd7518ef8f8e81b3b895b1ff4fe6e8a20915387d1f127edefccdcc3d867d 2024-08-20T22:53:54.0816649Z deleted: sha256:ab913355e8147522ebc4f29618812090844aa39956668afc4b91926827c327b3 2024-08-20T22:53:54.0817474Z deleted: sha256:69527fc1954cb8778833d15c0f3562fda1cbd230b11e649a974752f7a726cbf9 2024-08-20T22:53:54.0818307Z deleted: sha256:1b2a52846c812d7caf91918531cb04230500b31c3f695e444c1885aad0777b71 2024-08-20T22:53:54.0819405Z deleted: sha256:40f793cdc6b0a26ec9af5ea7188ad0ef19799a234e53eb4b8f3f756694a1000e 2024-08-20T22:53:54.0820298Z deleted: sha256:39b098ef95425e2d1bdb44559eda25dfc7bd62af2fb6bb691b540741278acb95 2024-08-20T22:53:54.0821258Z deleted: sha256:bc116c6279a58082c4ef9dac27e9d4d76bfdd78c2f2fe4439663d3dd2af55b57 2024-08-20T22:53:54.0822133Z deleted: sha256:57046a92505d1125e1af1100aaeb0e36175039b7dd36e67e7b4c554997d81b07 2024-08-20T22:53:54.0822966Z deleted: sha256:e868a8284f40af2175a17211327660844c1ac66ee31919f52c5e4e2b9564213a 2024-08-20T22:53:54.0823842Z deleted: sha256:ba5f7b6add2ae68d2962f19c3d981adc39633683af6dc3392c354cce87b7e8f9 2024-08-20T22:53:54.0824715Z deleted: sha256:3a06302b80c87d1030f701f23861d185613a3a4269faf533bb1b82b092a82887 2024-08-20T22:53:54.0825581Z deleted: sha256:dbdee317045b3156074f52d1fb32d613a48b6d8ed13c3e400a9f2c7bcceaf808 2024-08-20T22:53:54.0826461Z deleted: sha256:9574f911b87c39bbbb08a0238b82fb53abdedad1f387beb999ce1d36f3845e98 2024-08-20T22:53:54.0827356Z deleted: sha256:15d49efcad797dff0aa23e3b025b50c1c961668637e4bd9dff2361eaf3405c74 2024-08-20T22:53:54.0828252Z deleted: sha256:1b9b7346fee7abbc7f5538eaa23548bd05a45abe8daf6794024be0c8ad7d60bb 2024-08-20T22:53:54.0850871Z 2024-08-20T22:53:54.0851104Z Total reclaimed space: 35.51GB 2024-08-20T22:53:54.0921643Z Post job cleanup. 2024-08-20T22:53:54.0979652Z Post job cleanup. 2024-08-20T22:53:54.1828086Z [command]/usr/bin/git version 2024-08-20T22:53:54.1882614Z git version 2.40.1 2024-08-20T22:53:54.1917879Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/05330fe8-bcfe-4e8c-93db-d11644322a62' before making global git config changes 2024-08-20T22:53:54.1919222Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:53:54.1922437Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:53:54.1959973Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:53:54.1993545Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:53:54.2306196Z Entering 'android/libs/fbjni' 2024-08-20T22:53:54.2358668Z Entering 'third_party/FP16' 2024-08-20T22:53:54.2409828Z Entering 'third_party/FXdiv' 2024-08-20T22:53:54.2460020Z Entering 'third_party/NNPACK' 2024-08-20T22:53:54.2509309Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:53:54.2560832Z Entering 'third_party/XNNPACK' 2024-08-20T22:53:54.2627669Z Entering 'third_party/benchmark' 2024-08-20T22:53:54.2679676Z Entering 'third_party/cpp-httplib' 2024-08-20T22:53:54.2730066Z Entering 'third_party/cpuinfo' 2024-08-20T22:53:54.2780479Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:53:54.2829965Z Entering 'third_party/cutlass' 2024-08-20T22:53:54.2889908Z Entering 'third_party/eigen' 2024-08-20T22:53:54.2940897Z Entering 'third_party/fbgemm' 2024-08-20T22:53:54.2993695Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:53:54.3040963Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:53:54.3090426Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:53:54.3146313Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:53:54.3195921Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:53:54.3245586Z Entering 'third_party/flatbuffers' 2024-08-20T22:53:54.3299313Z Entering 'third_party/fmt' 2024-08-20T22:53:54.3348374Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:53:54.3399580Z Entering 'third_party/gloo' 2024-08-20T22:53:54.3448883Z Entering 'third_party/googletest' 2024-08-20T22:53:54.3500334Z Entering 'third_party/ideep' 2024-08-20T22:53:54.3547827Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:53:54.3604931Z Entering 'third_party/ittapi' 2024-08-20T22:53:54.3655692Z Entering 'third_party/kineto' 2024-08-20T22:53:54.3704137Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:53:54.3754973Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:53:54.3805822Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:53:54.3855120Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:53:54.3904210Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:53:54.3953147Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:53:54.4012671Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:53:54.4061359Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:53:54.4110040Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:53:54.4161702Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:53:54.4213603Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:53:54.4261471Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:53:54.4319696Z Entering 'third_party/mimalloc' 2024-08-20T22:53:54.4369809Z Entering 'third_party/nccl/nccl' 2024-08-20T22:53:54.4420472Z Entering 'third_party/nlohmann' 2024-08-20T22:53:54.4469814Z Entering 'third_party/onnx' 2024-08-20T22:53:54.4535328Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:53:54.4585020Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:53:54.4640016Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:53:54.4692147Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:53:54.4740721Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:53:54.4791793Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:53:54.4842678Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:53:54.4893181Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:53:54.4940913Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:53:54.4990311Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:53:54.5039002Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:53:54.5098986Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:53:54.5147922Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:53:54.5218127Z Entering 'third_party/pocketfft' 2024-08-20T22:53:54.5268410Z Entering 'third_party/protobuf' 2024-08-20T22:53:54.5321692Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:53:54.5372432Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:53:54.5425828Z Entering 'third_party/psimd' 2024-08-20T22:53:54.5479756Z Entering 'third_party/pthreadpool' 2024-08-20T22:53:54.5532792Z Entering 'third_party/pybind11' 2024-08-20T22:53:54.5581444Z Entering 'third_party/python-peachpy' 2024-08-20T22:53:54.5634038Z Entering 'third_party/sleef' 2024-08-20T22:53:54.5684786Z Entering 'third_party/tensorpipe' 2024-08-20T22:53:54.5734960Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:53:54.5782626Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:53:54.5833844Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:53:54.5888347Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:53:54.5939583Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:53:54.6007680Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:53:54.6035292Z http.https://github.com/.extraheader 2024-08-20T22:53:54.6042147Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T22:53:54.6076082Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:53:54.6354808Z Entering 'android/libs/fbjni' 2024-08-20T22:53:54.6386686Z http.https://github.com/.extraheader 2024-08-20T22:53:54.6419888Z Entering 'third_party/FP16' 2024-08-20T22:53:54.6454990Z http.https://github.com/.extraheader 2024-08-20T22:53:54.6487375Z Entering 'third_party/FXdiv' 2024-08-20T22:53:54.6521305Z http.https://github.com/.extraheader 2024-08-20T22:53:54.6552898Z Entering 'third_party/NNPACK' 2024-08-20T22:53:54.6585984Z http.https://github.com/.extraheader 2024-08-20T22:53:54.6619319Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:53:54.6651135Z http.https://github.com/.extraheader 2024-08-20T22:53:54.6683153Z Entering 'third_party/XNNPACK' 2024-08-20T22:53:54.6716398Z http.https://github.com/.extraheader 2024-08-20T22:53:54.6766625Z Entering 'third_party/benchmark' 2024-08-20T22:53:54.6802551Z http.https://github.com/.extraheader 2024-08-20T22:53:54.6840460Z Entering 'third_party/cpp-httplib' 2024-08-20T22:53:54.6875879Z http.https://github.com/.extraheader 2024-08-20T22:53:54.6905845Z Entering 'third_party/cpuinfo' 2024-08-20T22:53:54.6940319Z http.https://github.com/.extraheader 2024-08-20T22:53:54.6974354Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:53:54.7007093Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7039214Z Entering 'third_party/cutlass' 2024-08-20T22:53:54.7073284Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7115840Z Entering 'third_party/eigen' 2024-08-20T22:53:54.7148184Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7181369Z Entering 'third_party/fbgemm' 2024-08-20T22:53:54.7216135Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7248328Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:53:54.7281928Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7314627Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:53:54.7346776Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7381777Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:53:54.7415766Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7463019Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:53:54.7497198Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7529439Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:53:54.7562809Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7596433Z Entering 'third_party/flatbuffers' 2024-08-20T22:53:54.7629051Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7662979Z Entering 'third_party/fmt' 2024-08-20T22:53:54.7697632Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7730694Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:53:54.7764862Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7796900Z Entering 'third_party/gloo' 2024-08-20T22:53:54.7831176Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7861013Z Entering 'third_party/googletest' 2024-08-20T22:53:54.7895020Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7926361Z Entering 'third_party/ideep' 2024-08-20T22:53:54.7960158Z http.https://github.com/.extraheader 2024-08-20T22:53:54.7999884Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:53:54.8032438Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8075077Z Entering 'third_party/ittapi' 2024-08-20T22:53:54.8110068Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8142777Z Entering 'third_party/kineto' 2024-08-20T22:53:54.8177569Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8210143Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:53:54.8242815Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8276815Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:53:54.8307697Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8341332Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:53:54.8375142Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8407548Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:53:54.8441963Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8481493Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:53:54.8515166Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8550577Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:53:54.8582543Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8622027Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:53:54.8657740Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8689888Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:53:54.8722665Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8756808Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:53:54.8788147Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8827760Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:53:54.8861032Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8894645Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:53:54.8928589Z http.https://github.com/.extraheader 2024-08-20T22:53:54.8960029Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:53:54.8992858Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9041479Z Entering 'third_party/mimalloc' 2024-08-20T22:53:54.9075036Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9109158Z Entering 'third_party/nccl/nccl' 2024-08-20T22:53:54.9142190Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9175390Z Entering 'third_party/nlohmann' 2024-08-20T22:53:54.9208450Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9241758Z Entering 'third_party/onnx' 2024-08-20T22:53:54.9277166Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9324929Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:53:54.9359583Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9390882Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:53:54.9424873Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9462467Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:53:54.9498640Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9540573Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:53:54.9574252Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9605181Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:53:54.9638875Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9668411Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:53:54.9700467Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9737533Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:53:54.9770749Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9804211Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:53:54.9838171Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9867822Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:53:54.9900800Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9932606Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:53:54.9966062Z http.https://github.com/.extraheader 2024-08-20T22:53:54.9996533Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:53:55.0027657Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0064778Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:53:55.0097843Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0133040Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:53:55.0166104Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0216404Z Entering 'third_party/pocketfft' 2024-08-20T22:53:55.0248375Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0279699Z Entering 'third_party/protobuf' 2024-08-20T22:53:55.0314327Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0348711Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:53:55.0381329Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0417546Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:53:55.0449807Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0484148Z Entering 'third_party/psimd' 2024-08-20T22:53:55.0518017Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0548931Z Entering 'third_party/pthreadpool' 2024-08-20T22:53:55.0581678Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0614655Z Entering 'third_party/pybind11' 2024-08-20T22:53:55.0648089Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0680196Z Entering 'third_party/python-peachpy' 2024-08-20T22:53:55.0714505Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0745101Z Entering 'third_party/sleef' 2024-08-20T22:53:55.0779278Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0812924Z Entering 'third_party/tensorpipe' 2024-08-20T22:53:55.0845513Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0878076Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:53:55.0909139Z http.https://github.com/.extraheader 2024-08-20T22:53:55.0941098Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:53:55.0974691Z http.https://github.com/.extraheader 2024-08-20T22:53:55.1006666Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:53:55.1039760Z http.https://github.com/.extraheader 2024-08-20T22:53:55.1068691Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:53:55.1100795Z http.https://github.com/.extraheader 2024-08-20T22:53:55.1132836Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:53:55.1165698Z http.https://github.com/.extraheader 2024-08-20T22:53:55.1281988Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T22:53:55.1299020Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T22:53:55.1304530Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:53:55.1305046Z ##[endgroup] 2024-08-20T22:54:01.3046976Z Cleaning up orphan processes